Methods
Resolving identity by rule
Make each same-entity or different-entity decision explicit, evidenced, reviewed and reversible.
Problem
Names, addresses and identifiers can refer to different entities, while one entity can appear under several names. An unexplained merge contaminates every value attached to the resulting record; an unexplained split hides related evidence. Identity should therefore be treated as a decision with stated scope and grounds. The decision may be revised when evidence changes, while its earlier effects remain traceable. No shared name or single matching score explains an individual case by itself.
Steps
State the identity question
Name the two records and the type of entity each purports to describe. Ask whether they denote the same entity at a specified time, not whether their strings look similar. Preserve source identifiers and the exact fields compared. Distinguish a corporate parent from a subsidiary, a person from a namesake, and a renamed entity from its predecessor where relevant.
List evidence on both sides
Collect direct identifiers, authoritative registers, dated relationships and contradictory attributes with source passages and observation dates. Treat missing attributes as missing, not as evidence of difference. Note when two sources copy one claim. Evidence for and against a match should survive, including observations that might later prove a prior decision mistaken.
Write the decision rule
Express why the evidence supports "same", "different" or "unresolved" for this case. State any threshold, exception or authoritative identifier used, with the rule's version and scope. A reviewer should be able to apply the rule to the cited facts without relying on the first decision maker's memory. Where evidence is insufficient, preserve separate identities and the unresolved relation.
Review and preserve reversal
Have an authorized reviewer inspect the rule, evidence and consequences for dependent values. Record the decision date and review outcome. When new evidence overturns a merge or split, create a superseding decision and identify affected records; do not erase the old one. Reversibility requires knowing which values and conclusions depended on the earlier identity assignment.
Checks
- The record pair, entity type and date are explicit.
- Supporting and opposing evidence remain available.
- Rule, version, review outcome and decision date are recorded.
- A reversal can identify dependent assertions.
Pitfalls
- Treating a common name as a unique identifier.
- Using a similarity score as the whole explanation.
- Deleting the history of a corrected merge.
A worked example
Two fictional records: Halden Pharmacies Ltd, with a registry number and a head office address, and Halden Pharmacy Limited, with no registry number and a street address near the head office. The question is whether they denote the same entity at the date of a contract.
The evidence for a match: similar names after normalizing the legal form, a nearby address, and a registry entry that lists the second address as a branch. The evidence against: the second record has a different telephone area code and a different website domain. The rule written for this case says that a branch listed in the registry of the entity counts as the same legal entity, and that differing web domains do not count against a match. Applying the rule gives the answer same, with the registry page cited. A reviewer disagrees with the rule, not with an unexplained decision, and can propose a different one.
Variations and edge cases
Some cases need a relation, not a merge. A subsidiary and its parent are different entities with a link, and merging them would combine their obligations. A former name and a current name are the same entity at different times. A franchise and a franchisor are related and distinct. The method has three outcomes: same, different and unresolved. An unresolved pair stays separate, with the open question recorded, and is not forced into either bucket by a default.
Rules should be versioned. A rule that accepted a shared telephone number as sufficient may later be tightened, and the record should show which version applied to each decision, so the effect of the change can be measured and the affected decisions reviewed.
Evaluating the rules
Entity resolution research stresses that pairwise accuracy is not enough. Decisions propagate through transitive closure, so one wrong link can merge two large clusters, and errors in a few clusters can matter more than the average. Evaluation should therefore include cluster-level measures, a sample of decisions checked by people, and a check of the consequences: how many dependent values moved when a merge was made. Open-source tools for probabilistic linkage make the weights inspectable, and blocking techniques determine which pairs are ever compared, which is itself a source of missed links.
How the method is checked
An independent reviewer should be able to apply the rule to the evidence and reach the same decision. If they cannot, the rule is ambiguous. Reversal should be rehearsed: take a past merge, declare it wrong, and confirm that the dependent values can be listed and re-attached. A system that cannot do this has made merges irreversible in practice, whatever its policy says.
References
- Binette and Steorts (2022). (Almost) All of Entity Resolution. arXiv:2008.04443.
- Papadakis et al. (2019). A Survey of Blocking and Filtering Techniques for Entity Resolution. arXiv:1905.06167.
- Winkler, W. (2006). Overview of Record Linkage and Current Research Directions. U.S. Census Bureau.
- Splink: open-source probabilistic record linkage. Documentation.
Found an error on this page? See corrections for what counts as one, how to report it and the dated log.
Enquiry
Test this method on one entity.
Name an entity where "Resolving identity by rule" would be checked in practice, with its sources and its readers. We reply with what a first record would cover and what it would need from you.
Or write to [email protected] in your own words.