Skip to content
Reliable Context
Read the paperRequest access

The problem

Every organization already fuses private and public information. Almost none of them can defend the result.

The fusion is real work and it is done well. What does not survive it is the provenance: by the time two sources have become one sentence in one document, the reader has the conclusion and no way back to the evidence.

The four guaranteesInspect the demonstration
Two identical white cubes on a pale grey surface, lit from one side, each casting a shadow that falls in a different direction.
the figure

Two objects of identical description, throwing contradictory shadows. Nothing in the picture tells you which light was on when each was taken, and that is the whole difficulty: the values look the same, and the conditions that produced them are gone.

Two sources can agree on a name and disagree on everything that matters. A document that prints one of them prints no way to tell which.

The general case

The fusion happens in a person, in a document, and that is where the provenance dies.

Contracts, filings and internal systems on one side; registries, regulators and publications on the other. An analyst reads both, reconciles them, and writes a line. The line is what circulates. The reconciliation is not written down anywhere, and within a week it is no longer recoverable, including by the analyst. Data provenance research exists to record exactly this derivation [1, 2].

the line the document carrieswhat the reader is givenwhat the reader would need to check it

The supplier is majority owned by its founding family.

A sentence, and a source named in a footnote.

The filing, the page, the date it said so, and whether the holding has since changed.

The counterparty has no adverse findings.

A negative, stated flatly, with nothing behind it.

Which lists were queried, in which version, on which day, and what each returned.

Total exposure to this counterparty is under the limit.

One number, in one cell, summed from somewhere.

The contracts that were summed, and the basis for treating them as one counterparty.

The program is in Phase 2.

One value, current as of nothing in particular.

The version of the record that was read, and the two sources that said otherwise.

None of this is a failure of diligence. The analyst did the reconciliation; the organization paid for it; it was probably right. The defect is that the document is the only artefact, and a document is a flat surface: it has room for the conclusion and no room for the state of the evidence that produced it.

The cost appears later, and to somebody else. A second analyst re-reads the same four sources because there is no way to tell what the first one read. A client asks where a figure came from. A regulator asks what was checked. Each of those is answerable only by reconstructing work that was already done and then discarded.

The current answer

Why retrieval-centred context makes it worse

The prevailing response is to make the corpus searchable and assemble context at the moment a question is asked [3]. Passages are ranked by resemblance to the question, the top few are used, and the rest are not. This is a fine way to find something. It is a poor foundation for a value somebody has to defend, and it fails in a specific way: a passage is selected because it looks like the question, not because it has standing to answer it. Resemblance is not authority. The most similar sentence in a corpus is often a restatement, a summary or a draft, and nothing in the ranking distinguishes those from the filing. Where a citation is attached to the answer rather than read out of the source, deployed generative search systems have been measured printing sentences that their own citations do not fully support [4].

Then the answer outlives the passage. The assembled context is transient by design; the conclusion drawn from it is written into a memo, a model or a decision, and the conclusion persists long after the passages that supported it have been discarded. What was retrieved for one question is not what will be retrieved for the same question next month, because the corpus moved, the index moved, or the question was worded slightly differently. The conclusion is stable and its support is not. Time-oriented database research separates the moment a fact held in the world from the moment the database learned it, because a record that must answer for its past needs both [5].

The sharpest failure is the one nobody sees. A retrieval step that finds nothing produces an empty context, and an empty context produces a confident sentence with no hedge in it[6], because there is no mechanism by which not finding something becomes a recorded fact. Nothing states what was searched, in what version, on what date, or what it returned. A year later there is no object to re-read: no artefact that says this is what we knew, this is what we could not establish, and this is the day we looked. Databases have long made the underlying choice explicit: under the closed-world convention, whatever is not stored is taken to be false [7]. A retrieval step makes that choice silently.

retrieval-centred contextquestion0.910.880.840.620.58passages ranked by similarityanswerpassages discardednothing states what was searched and not foundthe same question assembles a different context next timea recordquestionone record, retainedownershipfiling, p. 14, read 11 Mar 2026exposuretwo contracts, both cited, 3 Feb 2026adverse findingsnot found, two lists searched 4 Feb 2026the same object, read again in 2027what was searched and not found is on the recordthe value and the evidence behind it stay attached
Two shapes. On the left, a pipeline that selects by similarity and keeps nothing. On the right, an object that retains what it read and holds the value against it.

Six properties, and what each of the two shapes is actually able to promise about them. The left-hand column is not a strawman: it is what a well-built retrieval system can honestly offer.

propertywhat a retrieval answer can promisewhat a record promisesverdict

Selection

That a passage will resemble the question closely, by whatever measure the index uses.

That a value is carried because a named source asserts it in a sentence.

resemblance is not standing

Provenance

That the passages used can be listed, for as long as the assembled context still exists.

That the excerpt, the locator and the observation date travel attached to the value.

listed, then gone

Absence

That a retrieval which found nothing can be reported as a retrieval that found nothing.

That a negative is declared, naming the sources searched and the day each was searched.

empty is not none

Lifetime

That the answer will persist. The context that produced it was transient by design.

That the value and its evidence have one lifetime and are retired together.

unequal lifetimes

Repetition

That the question can be asked again. Not that it will assemble the same context.

That the same object is read again, and states when any of it last changed.

not reproducible

Inspection

That the answer can be re-read a year later. What supported it cannot.

That there is one artefact, and it is the thing that was sent.

nothing to reopen

Three failure stories

The same three defects, in every organization that does this work.

Written generically, because they are generic. Each one is cheap to produce, expensive to discover, and invisible in the artefact that carries it.

What the record guarantees
01

Absence read as a negative

A query returned nothing, and the empty result left the building as a finding.

  1. An analyst screens a counterparty against sanctions and debarment lists before an award is signed.
  2. Two of the three lists return no rows. The third is unreachable that morning and is skipped.
  3. The cell in the working spreadsheet is filled in with None, because a spreadsheet has no other way to say it.
  4. The brief sent to the client states that the counterparty has no adverse findings.
  5. Nobody wrote down which lists were queried, which version of each, on which date, or that one was never reached.
  6. Eight months later the award is challenged and the only honest answer to what was checked is that nobody knows.
the defective output
adverse findings
NoneNo source. An empty result set, retyped as a finding.
lists searched
Not statedThe working file has no cell for it, so the information was never written anywhere.
list not reached
Omitted silentlyThe third list was skipped and left no trace in the output of any kind.
date of the check
AbsentThe brief carries its own date. Nothing ties that date to when any list was read.
what the record emits instead
adverse findings
Not foundTwo lists returned no rows when searched on 4 February 2026. A negative about the search, not about the counterparty.
lists searched
Two of three, each namedConsolidated sanctions list at version 2026.02 and the national debarment register at version 2026.01, both searched 4 February 2026.
list not reached
BlockedThe third register was unreachable on 4 February 2026. It is named, the reason is recorded, and the block is dated.
date of the check
4 February 2026Recorded against each list separately, because they were not all read at the same moment.

what the difference costsThe officer defending the award eight months later can say which lists were read, in which version, on which day, and which one was never reached. Holding the left-hand file, that officer can say nothing at all, and has to concede the whole screen rather than the one gap in it.

What the record does instead

An empty result set is recorded as a declared not-found carrying what was searched and when, so it can never be read as a negative about the world.

02

Two records merged on a name

Two real companies shared a trading name, and an ingest step folded them into one row.

  1. A supplier in one jurisdiction and a supplier in another trade under the same name and share a director.
  2. An intake step matched on the name, found it close enough, and attached the second company's documents to the first.
  3. From then on, a contract belonging to one counterparty sat on the record of the other.
  4. The exposure shown against the surviving record was the sum of two parties, and the arithmetic was faultless.
  5. The figure was not wrong about arithmetic. It was wrong about the world, and nothing in the output said so.
  6. The relationship between the two companies was real and worth stating. Merging them destroyed the ability to state it.
Two cast white plaster keys lying side by side on a pale surface, identical in outline but cut with different teeth, one of them fitted with a cobalt blue collar.

Two keys with the same outline and different cuts. Anything that matches on the silhouette will call them one key, and the mistake is only discovered by the person who has to open the door.

the defective output
counterparty
One row, two companiesMatched on the trading name during intake. No evidence of the match was retained.
exposure
One figure, summed across bothArithmetic correct, subject wrong. The total belongs to no counterparty that exists.
shared director
Used as the reason, then discardedIt justified the merge inside the intake step and appears nowhere in the output.
trading name
Treated as the identifierWhichever spelling arrived first became the key everything else attached to.
what the record emits instead
counterparty
Two records, one per companyRegister extract for each, read 6 March 2026, with the registration number that distinguishes them.
exposure
One figure per counterpartyEach summed only from the contracts cited on its own record, with the execution date of each.
shared director
Stated as a relation between two recordsBoth filings, quoted, with the date each was read. The connection is reportable without either record absorbing the other.
trading name
Retained on both records as a nameEach name kept with its source and observation date. A name is evidence about identity and never identity.

what the difference costsThe credit officer asked why a limit was breached now has to prove that two companies are two, after every report already sent has shown them as one, which moves the burden of proof onto the person who is right.

What the record does instead

Two things that share a name stay two records, related to each other with the basis of the relation written down, because a name is evidence about identity and never identity itself.

03

A value restated with no history

A source was re-read, the value was overwritten in place, and March stopped existing.

  1. A figure is read from a supplier's published statement in March and carried into a document sent to a client.
  2. In July the statement is revised, the source is re-read, and the system replaces the value where it stands.
  3. In September the client asks how the March figure was arrived at.
  4. The record shows the July value. The page it was read from now shows the revision.
  5. There is no way to establish what the record held in March, and therefore no way to say whether the March document was right when it was sent.
  6. The correction and the error have become indistinguishable, because only one of them is still visible.
the defective output
the figure
The July value onlyWritten over the March value in place. No earlier entry survives the update.
as it stood in March
UnavailableThe March state was not archived. It was the same cell, and the cell now holds something else.
the source page
Now shows the revisionThe citation points at a live page. The page that was actually read no longer exists.
what changed
Not statedA correction and an error look the same from here, because only one of them is still on the record.
what the record emits instead
the figure
The March value and the July valueTwo entries. The first read 6 March 2026, the second 9 July 2026, each with the statement it was read from.
as it stood in March
ReproducibleThe statement as it read on 6 March 2026 is retained, and the record can be read back at that date.
the source page
Both versions retained as readThe 6 March 2026 copy and the 9 July 2026 copy are both held, and each excerpt is verified against its own copy.
what changed
The July entry supersedes the March entrySupersession recorded 9 July 2026, naming what it replaces and leaving the replaced entry readable.

what the difference costsThe analyst asked in September whether the March brief was right when it was sent can answer the question from the right-hand record. From the left-hand one, the only available answer is that the record no longer contains March.

What the record does instead

A re-read writes a second entry rather than replacing the first, so the document sent in March can still be reproduced in September with the source each line rested on then.

The obvious objection

Why it is not solved by being careful

Every organization that produces this work already has careful people in it, and none of the three defects above is produced by carelessness. They are produced by the artefact. A document records a conclusion. It has no cell for the four lists that were queried, the one that was unreachable, the version each was read in, or the day the reading happened. The person who did that work has nowhere to put it, so it is not written down, and that is a property of the format rather than of the person. Care raises the quality of the conclusion. It cannot add a property to the object that carries it.

The second half is timing. A defect of provenance is not visible at the moment it is made, only at the moment it is questioned, which is usually months later and usually in front of somebody who was not there. In between, nothing distinguishes a line that can be defended from a line that cannot: both are sentences, of the same length, in the same font. Care applied at the moment of writing cannot see the difference either, because the difference is not in the sentence.

  1. 01

    The defect is invisible in the artefact that carries it.

    A sentence resting on a filing and a sentence resting on a recollection are the same sentence. A document has no place to put the difference, so re-reading the document, however carefully, cannot find it.

  2. 02

    The person who can see it is not the person who made it.

    The analyst knows which four sources were read and has nowhere to write it down. The reader has the place to write it down and does not know it. The knowledge and the slot for it sit in different heads.

  3. 03

    The cost lands months later, on somebody else.

    The bill arrives when a finding is challenged, in front of a regulator, a client or a court, and is charged to whoever holds the file then. The feedback never reaches the moment where care could have been applied.

What it costs

Four bills, and only one of them is visible.

The first is paid every week and is usually mistaken for the cost of doing the work. The other three arrive at the worst possible moment and are charged to somebody who was not in the room. We do not have an industry figure for any of these and we are not going to invent one.

  • rework

    The same source is read again by the next person, because nobody can tell what the last one read or whether it still holds.

  • correction

    A number already acted on has to be withdrawn, and the withdrawal is more expensive than the finding was worth.

  • re-examination

    A decision cannot be reopened on its merits, because the state of the evidence on the day it was taken no longer exists.

  • defence

    A finding put to a regulator, a court or a counterparty cannot be supported line by line, and the whole file is discounted rather than the one line.

References

References

The literature this page leans on: data provenance, retrieval-augmented generation, the measured gap between citations and the sentences they accompany, temporal databases and the closed-world convention.

  1. [1]

    Why and Where: A Characterization of Data Provenance

    Peter Buneman, Sanjeev Khanna and Wang-Chiew Tan

    International Conference on Database Theory (ICDT), 2001. https://doi.org/10.1007/3-540-44503-X_20

  2. [2]

    PROV-DM: The PROV Data Model

    Luc Moreau and Paolo Missier, editors

    W3C Recommendation, 30 April 2013. https://www.w3.org/TR/prov-dm/

  3. [3]

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen-tau Yih, Tim Rocktaschel, Sebastian Riedel and Douwe Kiela

    Advances in Neural Information Processing Systems (NeurIPS), 2020. https://arxiv.org/abs/2005.11401

  4. [4]

    Evaluating Verifiability in Generative Search Engines

    Nelson F. Liu, Tianyi Zhang and Percy Liang

    Findings of EMNLP, 2023. https://aclanthology.org/2023.findings-emnlp.467/

  5. [5]
  6. [6]

    Survey of Hallucination in Natural Language Generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto and Pascale Fung

    ACM Computing Surveys 55(12), 2023 (preprint arXiv:2202.03629). https://arxiv.org/abs/2202.03629

  7. [7]

    On Closed World Data Bases

    Raymond Reiter

    In Logic and Data Bases, Plenum Press, 1978, pages 55-76. https://doi.org/10.1007/978-1-4684-3384-5_3

Further reading

Articles on this page’s subject.

The research library

The remedy is an object, not a better search.

One canonical record per real thing, a cited and dated source behind every value, unknowns declared rather than blank, and any past state reproducible.

The four guaranteesHow the product worksRequest access