Glossary
Record linkage
Record linkage compares records to identify pairs that refer to the same entity despite imperfect identifiers.
Identity and entity resolution
Record linkage is the comparison of records to decide whether pairs describe the same entity. It is especially useful where a shared, reliable identifier is absent. The classical formulation separates links, nonlinks and pairs requiring further review. Comparison evidence may include names, dates and addresses, but no individual field is automatically decisive across every domain.
A practical linkage project defines the target entity and constructs candidate pairs. It then compares fields, evaluates disagreements and measures errors against reviewed examples. The cost of a false link often differs from the cost of a missed link, so decision thresholds should reflect the task. Blocking can reduce the number of comparisons, but it can also hide true links.
For a defensible record, linkage needs an inspectable basis for each consequential pair decision. Merely reporting a similarity score conceals what matched, what differed and which source supplied each field. Entity resolution names the wider identity problem; the Fellegi–Sunter model supplies one statistical framework for pair classification; canonical identifiers label the resulting entities.
Example
Two Halden Therapeutics AG (fictional) records share a registration number despite different abbreviations; the pair is reviewed as a likely link.
From census matching to administrative data
Record linkage has a long history in public health and official statistics, where the task is to link records of the same person across registers that share no reliable identifier. The statistical theory was set out by Fellegi and Sunter in 1969, building on earlier work by Newcombe and others, and the methods were developed and applied extensively at statistical agencies; the working paper by Winkler (2006) of the US Census Bureau surveys the techniques and their practical refinements, such as approximate string comparators and handling of frequency of values. Modern implementations of the same theory are available in open software, and the Splink documentation explains the model in applied terms.
The word linkage emphasises connecting records across datasets or within one, with an assessment of how likely each pair is to be a true match. It differs in emphasis from entity resolution, which includes the later steps of grouping and identifier assignment, though the terms are used interchangeably in practice. Social scientists use probabilistic methods for linking large administrative files, as in the fastLink approach of Enamorado and colleagues, which adapts the Fellegi–Sunter framework to very large files and to uncertainty about the links.
Practical design choices
A linkage design makes several choices that determine its error profile. Which fields are compared, and how: exact match for identifiers, edit distance or phonetic codes for names, tolerance for dates. How missing values are treated: a missing field should neither support nor oppose a match. How the frequency of values is used: agreement on a rare surname is stronger evidence than agreement on a common one. Where thresholds are set: a high threshold yields few false links and more missed ones, and the right balance depends on the cost of each error for the use. And whether a clerical review band exists for the pairs in the middle, which is where human judgment is most valuable and most expensive.
Linkage quality is estimated, not assumed. Draw a random sample of linked pairs and of unlinked candidate pairs, have them reviewed by people, and report the error rates with their uncertainty. Repeat after any change in the source data or the method. The article on evaluating entity resolution beyond pairwise F1 explains why a single score from a clean test set tends to overstate performance on production data.
A worked example
Two files list contacts at a fictional customer of Halden Therapeutics AG. File one has full names, an employer and a work email domain. File two has initials, a surname, a city and a phone prefix. Exact matching on name finds almost nothing, because of initials. Blocking on the first three letters of the surname and the city produces a manageable set of candidate pairs. Comparison scores surname agreement with a string measure, initial agreement as a binary field, and city agreement as exact. A pair with a rare surname, matching initial and matching city receives a strong positive total weight and is linked; a pair with a common surname and matching initial but a different city receives a total near zero and goes to review. The review log of these middle cases becomes the training material for the next version of the rules.
Related terms and boundaries
Linkage is the pairwise core of entity resolution. The Fellegi–Sunter model supplies its probabilistic scoring, blocking its scalability, and a canonical identifier its output. It is separate from entity disambiguation, where one side of the comparison is free text, and from schema alignment, which must usually be settled before the fields can be compared at all.
Checks a reviewer can run
Ask for the comparison fields and how each is scored. Ask how missing values are treated. Sample linked and unlinked pairs for review and report error rates with intervals. Ask how the clerical review band is staffed and logged, and when the rules were last tuned.
Common misreadings
- Using a universal threshold across different populations.
- Ignoring asymmetric costs of false and missed links.
- Assuming blocking cannot remove true matches.
Appears in
- Deterministic vs probabilistic entity resolution
Deterministic rules make explicit match decisions; probabilistic methods estimate match likelihood from observed features.
- Resolving identity by rule
Make each same-entity or different-entity decision explicit, evidenced, reviewed and reversible.
References
- Fellegi and Sunter (1969), A Theory for Record Linkage, Journal of the American Statistical Association 64(328).
- Winkler (2006), Overview of Record Linkage and Current Research Directions, US Census Bureau.
- Splink documentation, The Fellegi-Sunter model.
- Enamorado, Fifield and Imai (2019), Using a Probabilistic Model to Assist Merging of Large-Scale Administrative Records, American Political Science Review 113(2).
- Wikipedia, Record linkage.
Found an error on this page? See corrections for what counts as one, how to report it and the dated log.
Enquiry
Where does "Record linkage" bear on a decision of yours?
This entry sits under identity and entity resolution. If a decision in your organization turns on it, send the entity concerned, the sources you fuse to describe it and who may read which of them. We reply with what a first record would cover.
Or write to [email protected] in your own words.