Glossary
Truth discovery
Truth discovery estimates which conflicting claims are credible when the correct value is not directly known.
Truth discovery
Truth discovery studies how to assess conflicting claims about the same subject and attribute when an authoritative answer is unavailable. The task begins only after claims have been assigned to the same subject, property and relevant time. A result may be a selected value, a ranking or an explicit unresolved state. Its evidential force depends on assumptions about sources and the claim space, not simply on how many records agree.
In practice, an analyst first preserves the competing claims and identifies their sources. Some methods infer source reliability and claim credibility jointly; others use known reference cases or domain constraints. Multiple values can be true for a multivalued property, while an apparently conflicting value may describe a different date. The problem formulation must therefore specify whether one value, several values or no supported value may be returned.
An organization defending a value must show which alternatives were considered, what evidence supports the chosen claim and where uncertainty remains. A numerical confidence estimate alone cannot substitute for that record. Truth discovery is narrower than knowledge fusion, which also reconciles entities and representations, and broader than majority voting, which counts claims without modelling the credibility or dependence of their sources.
Example
Halden Therapeutics AG (fictional) sees three different registration statuses for the same trial on the same date; the claims remain separate until their sources and dates are assessed.
How the problem is set up
The input to a truth-discovery method is a table of claims. Each row names a source, a subject, a property and the value the source reports, often with a date. The output is, for each subject and property, a selected value or a ranked list of values, usually together with an estimate of how far each source can be relied on. The survey by Li and colleagues organises the literature by the assumptions a method makes: whether sources are treated as independent, whether more than one value can be true, whether the property is categorical or numeric, and whether the data arrive in a batch or as a stream.
Most methods alternate between two estimates. Given current beliefs about which values are correct, they re-estimate how accurate each source has been; given current beliefs about source accuracy, they re-estimate which values are correct. TruthFinder is the best-known example and the Dawid–Skene model is the same idea applied to categorical labels from human annotators. Methods differ in how they stop, how they start, and what they do when the iteration has more than one stable answer. The practical consequence is that two reasonable methods can disagree on the same claims, which is why the result of any one of them is a documented aggregation and not a discovery of fact.
Web-scale systems add a further complication: the claims themselves are extracted by software. The Knowledge-Based Trust work uses a multi-layer probabilistic model so that a mistake made by an extractor is not blamed on the page it read, and Knowledge Vault combines extractions with prior knowledge from existing knowledge bases. Both show that the question of which source to believe cannot be separated from the question of whether the claim was read correctly.
A worked example
Halden Therapeutics AG (fictional) is described by five sources as to the start date of a trial. A regulatory registry and the sponsor's own page give 4 March 2025. Three aggregator pages give 11 March 2025, and all three reproduce, nearly word for word, a press release that predates the registry update. Counting sources gives 3 to 2 for 11 March. Counting original observations, with the aggregators folded into one lineage with the press release, gives 2 to 1 for 4 March. A method that models source dependence reaches the second result by design; majority voting reaches the first unless the analyst deduplicates beforehand.
Even the second answer is conditional. The press release may have stated a planned date, and the registry an actual one, in which case the two claims do not conflict at all; they describe different events. A defensible record would therefore keep both claims with their wording, carry the registry date as the start date with the stated basis, and store the press-release date as a planned date. The aggregation step is useful, but the alignment of subject, property and time has to come first, as the entry on data conflict explains.
How to evaluate a method before relying on it
A method should be tested on cases whose answers are known and which resemble the cases it will meet. The comparison that matters is against simple baselines: a method that cannot beat voting on the organisation's own data has not earned its complexity. Results should be reported by property and by source group, since a method that is accurate on identifiers may be poor on dates or statuses. Cases in which every source is wrong deserve separate attention, because most methods can only choose among the values they are given; this is the subject of the article on truth existence in data fusion.
Where no labelled answers exist, indirect checks are still available: stability of the result under removal of a source, sensitivity to injected copies, and agreement with a small sample adjudicated by an expert. The articles on testing truth-discovery algorithms and on comparing methods without gold labels set out protocols for this. Whatever the method, retain the full set of alternatives considered so that a reviewer can see what the result was chosen from, and treat an explicit unresolved state as a legitimate output.
Where truth discovery stops
Truth discovery assumes that a claim has already been attached to the right subject and the right time. If the subject is uncertain, the first task is entity resolution; if the property is ambiguous, it is schema alignment. It also does not establish that anything is true in the sense of being verified against the world. It reports which claim is best supported given the evidence and assumptions, and the strength of the support is the thing a decision-maker needs. The broader activity of assembling a coherent body of statements from many sources is knowledge fusion, of which truth discovery is one component, and the stored form of its inputs is the claim.
Checks a reviewer can run
Ask which method produced a given value, with what parameters, and what the result would have been under simple voting. Ask for the alternatives that were considered and their sources, not only the winner. Remove one source and rerun to see how stable the answer is. Insert a copied source and see whether the result moves. Ask which cases ended unresolved and who reviewed them. Sample twenty results and trace each to the claims behind it, confirming that the subject, property and date were aligned before aggregation.
Common misreadings
- Treating repeated copies as independent confirmation.
- Comparing claims about different dates as if they conflict.
- Forcing a single answer when the property permits several values.
Appears in
- Majority voting vs truth discovery
A count of agreeing sources is a useful baseline; truth discovery also considers how those sources behave.
- Estimating source reliability
Assess a source for a defined property and period using checked outcomes and visible uncertainty.
- Reporting source conflicts
Preserve competing values and their grounds until a scoped decision can be defended.
References
- Li et al. (2016), A Survey on Truth Discovery, ACM SIGKDD Explorations 17(2).
- Dong et al. (2015), Knowledge-Based Trust: Estimating the Trustworthiness of Web Sources, PVLDB 8(9).
- Dong et al. (2014), Knowledge Vault: A Web-Scale Approach to Probabilistic Knowledge Fusion, KDD.
- Raykar et al. (2010), Learning From Crowds, Journal of Machine Learning Research 11.
Found an error on this page? See corrections for what counts as one, how to report it and the dated log.
Enquiry
Where does "Truth discovery" bear on a decision of yours?
This entry sits under truth discovery. If a decision in your organization turns on it, send the entity concerned, the sources you fuse to describe it and who may read which of them. We reply with what a first record would cover.
Or write to [email protected] in your own words.