Comparisons
Deterministic vs probabilistic entity resolution
Deterministic rules make explicit match decisions; probabilistic methods estimate match likelihood from observed features.
The distinction
Deterministic entity resolution applies explicit conditions, such as equality of a trusted identifier or a defined combination of fields. It is easy to explain which rule fired and can be appropriate where identifiers have strong governance and false merges are costly. Its limits appear when identifiers are missing, mistyped, reused, or issued at different levels of an organization. A rule can also be consistently wrong if the input field does not mean what the rule assumes it means.
Probabilistic entity resolution weighs agreement and disagreement across identifying fields to estimate whether records refer to the same subject. The Fellegi-Sunter framework is a foundational account of record linkage under uncertainty. Such methods can surface plausible matches despite incomplete or noisy data, with thresholds for matches, nonmatches, and review. Their scores depend on data quality, population, and modeling assumptions; a high score is not a universal identity guarantee. Both approaches require a policy for review, correction, and preserving decisions over time.
By dimension
| Dimension | Deterministic resolution | Probabilistic resolution |
|---|---|---|
| Decision basis | Specified predicates over fields. | Estimated evidence for match and nonmatch. |
| Missing identifiers | May leave a record unmatched. | Other identifying features may still support a candidate. |
| Explanation | The triggering rule is direct to inspect. | Field contributions and threshold choices need explanation. |
| Threshold | A rule boundary gives a categorical result. | Scores can support match, nonmatch, and review bands. |
| Error exposure | Brittle rules miss variants or merge on misleading fields. | Poor calibration can yield false matches or false nonmatches. |
| Data requirement | Relies on fields with clear semantics. | Needs suitable comparison data and model assessment. |
| Governance | Rule versions and exceptions must be retained. | Model versions, thresholds, and adjudications must be retained. |
Choose Deterministic resolution when
- A trusted, scoped identifier is available.
- A decision must follow a clearly stated business rule.
- The volume of ambiguous cases is manageable for review.
Choose Probabilistic resolution when
- Identifiers are incomplete and fields contain noise.
- Several weak signals together carry useful evidence.
- Uncertain candidates can be routed for adjudication.
Using both
Apply high-confidence deterministic rules where their premises hold, use probabilistic candidates for unresolved cases, and retain the basis of every final identity decision.
A worked example
Two fictional files list a pharmacy chain. One carries a registry number and the name Halden Pharmacies Ltd. The other carries no registry number and the name Halden Pharmacy Limited, with an address one street away. A deterministic rule that requires equal registry numbers does not fire, so the records stay apart.
A probabilistic model compares name, address, phone and postcode, assigns weights to agreement and disagreement on each, and combines them into a score. The pair scores in the uncertain band between the thresholds, and goes to a reviewer. The reviewer finds the second file's address in the registry entry's list of branches and records a match, citing the entry. The deterministic rule would have missed the link. The probabilistic score alone would not have settled it. The reviewer's decision, with its evidence, is what the record keeps.
Where each one fails
Deterministic rules fail on dirty or missing identifiers and on identifiers that do not mean what the rule assumes. A shared phone number linking two unrelated businesses is a classic example. A rule that is simple to explain can be consistently wrong, and its failures are invisible unless someone samples them.
Probabilistic methods fail on assumptions. The classic framework assumes that fields agree independently given the match status, which is false for correlated fields such as street and postcode. Weights estimated on one population may not transfer to another. Scores drift when the data changes. Modern implementations make it easier to train and inspect models, and the literature on evaluation stresses measuring performance beyond pairwise agreement, since errors cluster and transitive closure can merge distinct entities.
Questions to settle before choosing
How reliable are the identifiers, and who governs them? What is the cost of a false merge compared with a missed link? How many ambiguous pairs can a team review? Is there a labelled sample from this population to set and test thresholds? Are decisions reversible, and are the effects of a merge traceable to the values it moved? Where false merges are very costly, rules with human review of the uncertain remainder are usually safer than a high-recall model.
How the two meet in a review
The usual arrangement layers them. Deterministic rules settle the clear cases and are cheap to audit. A probabilistic model ranks the rest. Reviewers decide the uncertain band. Every final decision carries its basis: the rule, the score and threshold, or the reviewer and evidence. When a decision is reversed, the record shows which values depended on it. Evaluation reports precision and recall for each layer, and for the complete pipeline, with a measure that is sensitive to cluster errors.
Related terms
References
- Fellegi and Sunter (1969). A Theory for Record Linkage. Journal of the American Statistical Association 64(328).
- Winkler, W. (2006). Overview of Record Linkage and Current Research Directions. U.S. Census Bureau.
- Binette and Steorts (2022). (Almost) All of Entity Resolution. arXiv:2008.04443.
- Splink: open-source probabilistic record linkage. Documentation.
Found an error on this page? See corrections for what counts as one, how to report it and the dated log.
Enquiry
Which do you run today: Deterministic resolution or Probabilistic resolution?
Describe how your organization handles this now and one entity where it falls short. We reply with where a first record would sit alongside it and what it would need from you.
Or write to [email protected] in your own words.