A kernel of truth: records that carry their evidence in organizations that fuse private and public information

Fusion is usually described as a merge: take a public record, take an internal record, produce one. The description hides where the difficulty is. The two halves do not fail in the same way. The public half is contestable by anyone and moves underneath you, because a registry is corrected, a filing is superseded and a page is edited without a diff; it is honestly citable only as it stood on a stated date. The private half is stable and often authoritative, and cannot be shown to the person disputing the result, because of counsel, because of a licence, or because it names somebody. A fused value therefore rests on two supports with incompatible availability, and that asymmetry, not the join, is the engineering problem.

The distinction worth making is between merging records and merging warrants. Merging records is a data problem with a mature toolkit. Merging warrants is the question of what a reader is entitled to conclude from the combination, and it has no automatic answer: the combined value is as contestable as its weakest public support and as unshowable as its most restricted private one. A system that treats the join as the whole job emits a value whose warrant nobody can state, and the first person to ask for it is usually the last person you want asking.

The tools exist. Database provenance gives precise accounts of why a tuple appears in a result and where each of its values came from ^[1], the semiring formulation shows that such accounts compose through the query algebra rather than being bolted on afterwards ^[2], and PROV gives a vocabulary for moving them between systems that were not built together ^{[3, 4, 5]}. None of it survives the step most pipelines take next, which is to summarise. A summary is a new artefact with no provenance of its own unless one is constructed for it deliberately, and once the summary is what the reader sees, the derivation underneath it is decorative. Provenance does not die at the join. It dies at the point where the system starts producing prose.

Two objections are fair. The first is cost: attaching a sentence, a locator and a retained copy to every value is more expensive than writing the value. It is, and the cost is paid once, at leisure, against a cost that otherwise falls at the worst possible moment. The second is harder. Some values are genuinely not attributable to a single sentence: a ratio, a total, a classification drawn across twelve documents has no quotable origin. The wrong response is to pick one input and cite it, which is this paper’s subject in its most respectable disguise. The right one is to record a derivation as a derivation, naming its inputs and their evidence, so a reader can descend one level and find sentences at the bottom. Records management has defined a record by its capacity to serve as evidence of an activity for decades ^[6]; a summary with no derivation does not meet that definition, and no amount of fluency makes it one.

Linkage was mechanised on vital records in the 1950s^[7] and formalised by Fellegi and Sunter as a decision procedure over agreement patterns with explicit error rates ^[8]. Most of the work since has made the comparison cheaper and the classifier better^{[9, 10, 22]}. All of it answers a question of degree: how alike are two records. The operational question is not a question of degree. It is whether these are one thing. A similarity threshold answers the first and is then used as though it had answered the second, which is why tuning a threshold feels like making policy. It is making policy, at a scale of hundredths, with no record that a policy was made.

A name is the most available comparison and among the least reliable, because names are not identifiers. They are labels administered by other people for their own purposes, and they are correct only relative to a document and a date. One company carries three names across a decade, an incorporation name, a trading name and a post-acquisition name, and every one of them is right on some filing. Two companies in different jurisdictions carry one name, one director and one registered agent, and are two companies.

The two failures that follow are not symmetric. A bad merge sums an exposure across two counterparties and produces a number that is wrong in a direction nobody checks, because the file looks more complete than it did before. A bad split hides a real connection, and is the error that survives an audit, because nothing about a split looks anomalous: two thin records are exactly what a young relationship is supposed to look like. The merge is louder when it is found. The split is almost never found.

Frequently the correct output is neither, but two records that remain two with the relation between them stated, dated and readable. That is unsatisfying to anyone who wants a clean count of entities, and it is the concession this section makes: a system that refuses to merge on insufficient evidence will show a reader two rows where the world has one thing, and will look wrong in precisely the way a careful system looks wrong. The defence is asymmetry again. Two rows can be joined later, with the evidence that justified the join attached to the act. A bad merge destroys the separation, and the evidence that would have undone it is now filed under the surviving identifier where nobody will think to look. What must not happen, in either direction, is that the judgement stays implicit in a coefficient. Whether two records are one thing is a claim about the world. It is either stated, evidenced and dated, or it is a side effect of a parameter somebody set on a Tuesday.

A blank cell is read as a negative by almost every reader, and it is not one. Three different states of the world are routinely rendered as the same emptiness: nothing was found after looking, something exists but this reader is not licensed to open it, and the property does not apply to this entity at all. The three are not variations on a theme. The first is a statement about a search, the second about a permission, the third about the kind of thing being described. Collapsing them destroys information the reader often needs more than the value itself, because each implies a different next action: widen the search, request clearance, stop asking.

Declaring the negative in words is cheap. Making the declaration worth anything is not, because a negative is a claim and a claim needs evidence of its own: which sources were searched, under what query, on what date, and what was found instead. Without that, a declared unknown is a blank with better manners. With it, the reader can disagree with the search rather than with the silence, which is the only form of disagreement that leads anywhere. It also changes what the value is: a negative supported by a dated search is a finding, and findings can be re-run.

Establishing that a declaration was correct is genuinely hard, and the honest position is that nobody has solved it. The ground truth is also an absence, so it cannot be annotated the way a positive can. Fact-verification benchmarks carry a not-enough-information class over a fixed corpus^[16], which measures something real but not this: a claim unsupported within one corpus is a different statement from a property that does not exist in the world. Treating the two as interchangeable is how a system trained to be careful learns to be confidently empty. Databases have long made the underlying choice explicit: the closed-world convention treats whatever is not stored as false ^[25], and a declared negative is the refusal to adopt that convention silently.

The objection from the other side is commercial rather than technical, and it is the serious one. A file full of declared unknowns looks worse than the same file with blanks. It is less saleable, it invites questions, and it makes the reader do arithmetic about coverage that the blank version quietly did for them. The answer is that the blank version was never more complete; it was less honest at the same completeness. There is also a consequence the blank cannot have: a declared negative acquires an expiry the day it is written, because a search that found nothing in March says nothing about June. A blank makes no claim, so it never goes stale, and that is exactly why it survives in files long after it stopped being true.

A record without time is a photograph presented as a law. Every value in a fused record carries at least three dates that matter and are routinely conflated: the date the source itself states, the date the source was read, and the date the value entered the record. Retrieval date standing in for stated date is the most common of these errors and the hardest to find afterwards, because the value is usually still right and nothing fails. It becomes visible only when the value stops being right, which is the one moment at which nobody has time to audit how it was derived.

Replay makes the distinction operational. If the record can be reconstructed as it stood on a past day, together with the sources that supported it on that day, a decision can be defended against the record its maker actually had, rather than against a record that has moved since. That is not a convenience. A decision judged against today’s evidence is judged against evidence its maker did not have, which is unfair when the decision was good and unfalsifiable when it was bad.

The temporal-database literature already separates the time a fact holds in the world from the time the database learned of it ^[23], and the SQL standard has carried both since 2011, as application-time periods and system-versioned tables ^[24]. Versioning a row is not replay, and the difference is where most implementations quietly fail. A store that keeps every past value, but refreshes its cached sources in place, can reconstruct what the record said and not what it knew. The excerpt still matches the copy on disk because the copy on disk was replaced. Replay requires that the evidence be immutable too, that a correction be an additional entry rather than a replacement, and that the superseded value stay readable with the reason it was superseded. Provenance vocabularies model derivation as timed activity and give a place to put all of this^[3]; keeping the times honest is a discipline, not a data type.

The objection is that this is expensive and that nobody asks for last March. Both halves are half true. Append-only storage of values and their sources costs real money, and the ergonomics are worse: an overwrite is one statement, a supersession is three, and every reader now has to be told which state they are looking at. But the people who ask for last March are precisely the people whose questions cannot be declined, being a regulator, opposing counsel, an acquirer’s diligence team or an internal investigation into a decision that went wrong. They ask rarely and they ask late, long after the analyst has left and the pipeline has been rewritten twice. A system that cannot answer them is not holding a record. It is holding a claim about what it currently believes, which is a different and much weaker object.

Placing retrieved passages beside a question improved the answers, and the improvement is not in dispute^[15]. Generated text that its source does not support is a documented failure mode of language models across tasks^[21]. The arrangement is the right default for a great many tasks, and nothing below is an argument for doing without retrieval. It is an argument about what the arrangement can be asked to promise, and the answer is: less than it is routinely asked to promise by people who then act on the result.

The first limit is attribution. There is a workable definition, namely whether a generated statement is supported by the source attached to it ^[17], and when generative search systems were measured against that definition, a substantial share of their sentences were found not to be fully supported by their own citations ^[18]. That is not a defect of one product. It is what happens when a citation is produced alongside an answer rather than being the thing the answer was read out of. The ordering is the whole of it: a citation chosen to accompany a sentence is a plausible source, and a sentence extracted from a source is an attributed one. From the outside the two look identical, which is why the difference has to be enforced where the value is written and cannot be recovered by inspecting the output.

The second limit is time. Retrieval returns what the index holds now, which is not what the analyst read in March, and an index has no obligation to remember that it once held something else. Nor does it have any way to tell you what it did not hold: an empty result set is indistinguishable from an unlucky query, and both are indistinguishable from a document that was removed. A retrieval-centred system can answer a question well today. On its own it cannot tell you what it answered last quarter, what it was reading when it did, or whether that source still says the same thing.

The fair rejoinder is that none of these limits are inherent, and that a retrieval stack can be made to retain what it read, timestamp it and keep the excerpt. That is exactly right, and it is the argument of this paper stated from the other end: the part that has to change is not the model or the index but what happens after the answer, where the retained copy, the excerpt and the dates either become part of a record or are discarded with the context window. A system that keeps them has stopped being a retrieval pipeline with citations and has started being a system of record, whatever it is called internally.

The reader who matters was not present when the value was written, has no way to derive it again, and is often not friendly. They arrive years later with a specific question about one value in a file somebody acted on, and their first move is to ask where it came from. Everything in this paper is an attempt to make that question answerable by the record itself rather than by whoever is still at the company.

Four things are owed to that reader, and each is a refusal as much as a promise. One canonical record per real thing refuses the convenience of letting two spellings become two companies, and the greater convenience of letting two companies become one. A verbatim, dated source behind every value refuses a source name standing in for a sentence, which is the form most citation actually takes. A declared unknown refuses the blank cell, at the price of admitting on the surface how much is not known. Replay refuses the overwrite, and with it most of the ergonomics of a mutable store. Each refusal is unpleasant in the moment it is enforced, which is why systems that do not enforce them mechanically do not enforce them at all.

It is worth being precise about what this does not buy. Truth-discovery and data-fusion methods estimate a value and the reliability of its sources jointly, and estimate them well^{[11, 12, 13, 14, 19, 20]}. Nothing here improves on them as estimation, and a system built on the properties above may well be less accurate than one that weights sources statistically. The distinction is between accuracy and defensibility. A reliability score cannot be read back to the sentence that produced it and cannot be argued with by the person it was used against, so it is the wrong instrument at the moment a value is challenged, however right it was on average across the corpus.

The promise, then, is smaller than truth, and stating it honestly matters more than stating it grandly. A record that carries its evidence is not thereby correct: its sources can be wrong, its readings can be shallow, and its declared negatives can be lazy. What it is, is accountable. Every value on it resolves to a sentence, a locator and a date, and therefore to something that can be confronted, re-read and overturned by a person who was not there. That is the whole of what a system of record can promise, and organizations that act on fused information have been making decisions for years on files that cannot promise even that.