alt Hacker NewsAnother great article, thanks. I thought a lot about that too, and basically realized the same thing (this wasn't the coding agent), that frequency in the dataset mattered a lot for how useful a match is, and made frequency lists from my datasets, both on how common the literal names were and the phoneme-reduced names.
But I'm worried about that "bitter lesson" the TypeSafe CEO refers to, and that we're retreading the steps of natural language processing and a lot of other fields, trying to come up with clever rules, when the rule-based approach simply never gets good enough. I need matching to be good, it's absolutely central to genealogy, and I've seen the damage blindly linking by hand-crafted matching formulas can do.
I agree, and I think LLMs can potentially do a better job than more conventional methods, so long as they're provided with enough content.
Another aspect of context that could be relevant to your work may be how people are nested within households. In some record linkage scenarios you can run a linkage on household membership and use this to make person linkages more precise. I imagine an LLM agent could also use this information effectively if it were explicitly provided, but sometimes they miss logical leaps like this