Data Provenance vs Data Lineage vs Audit Trail

The same value, three questions

Pick any value a regulated organization acts on: a supplier's certificate expiry date, a counterparty's legal name, a batch's assay result. Three different questions can be asked about it, and they are often confused:

1. Which datasets and jobs produced the table this value sits in? That is data lineage.
2. Where did this particular value come from, how was it derived, and who is responsible for it? That is data provenance.
3. Who created, changed or deleted this record, and when? That is an audit trail.

Each question has its own standards, its own tools and its own regulatory sources. A data catalog that draws lineage graphs does not answer the second question. An audit trail does not answer the first. Provenance records rarely answer the third. The terms overlap at the edges, but treating them as synonyms leads to controls that look complete on a diagram and fail in an inspection.

This article defines each term against a published standard or regulation, follows one fictional value through all three, and sets out where each fails when used alone.

The worked example

Halden Therapeutics AG (a fictional company) keeps a supplier master table. One row describes Velmar Chemie GmbH (also fictional), an approved supplier of an active ingredient. The field of interest is:

A quality dashboard flags suppliers whose certificate expires within 90 days. On 28 October 2026 it shows Velmar as compliant. An inspector asks how Halden knows the date is right. The three disciplines give three different answers.

Data lineage: datasets, jobs and runs

Data lineage is the record of how data moves and is transformed between datasets. NIST's glossary, quoting its Privacy Framework, defines lineage as "the history of processing of a data element, which may include point-to-point data flows and the data actions performed upon the data element" ^[1]. In practice, enterprise lineage tools work at the level of tables, files and jobs, sometimes columns.

OpenLineage, an open specification for collecting lineage metadata, is a clear example of the model. Its object model has three core entities ^[2]:

• A Job is "a process that consumes or produces Datasets."
• A Run is "an instance of a Job that represents one of its occurrences in time", identified by a runId.
• A Dataset is "an abstract representation of data", identified by a name within a namespace "derived from its physical location (i.e., db.host.database.schema.table)."

These can be extended with facets, defined as "an atomic piece of metadata identified by its name", attached to runs, jobs or datasets ^[3].

For Halden, the lineage record says:

This is useful. It tells an engineer that if qa.certificates was wrong, supplier_master is wrong too, and which run to re-execute. It does not say where the date 2027-05-31 came from before it entered qa.certificates, which document supports it, or who typed it in. Lineage describes the plumbing, not the evidence.

Data provenance: source, derivation and responsibility

Data provenance is the documented origin and history of a specific piece of data, including who was responsible for it. The W3C PROV Data Model, a W3C Recommendation of 30 April 2013, defines provenance as "a record that describes the people, institutions, entities, and activities involved in producing, influencing, or delivering a piece of data or a thing" ^[4]. NIST's glossary gives a security-oriented definition: "the chronology of the origin, development, ownership, location, and changes to a system or system component and associated data" ^[5].

PROV-DM has three core types ^[4]:

• An entity is "a physical, digital, conceptual, or other kind of thing with some fixed aspects".
• An activity is "something that occurs over a period of time and acts upon or with entities".
• An agent is "something that bears some form of responsibility for an activity taking place, for the existence of an entity, or for another agent's activity".

They are connected by relations including generation, usage, derivation ("a transformation of an entity into another, an update of an entity resulting in a new one, or the construction of a new entity based on a pre-existing entity"), attribution ("the ascribing of an entity to an agent") and association.

For Halden, the provenance of the value is:

This answers the inspector's question directly: the date comes from a specific certificate, read on a specific day by a named person, with the passage that supports it. The live article on minimum data provenance requirements sets out which of these fields are needed for a value to be defensible.

Provenance has a limit too. It records how the value came to be what it is. On its own, it does not show whether anyone later overwrote the field without updating the provenance record.

Audit trail: who changed what, and when

An audit trail is a chronological record of actions on a system or record. NIST defines it as "a chronological record that reconstructs and examines the sequence of activities surrounding or leading to a specific operation, procedure, or event in a security-relevant transaction from inception to final result" ^[6].

In regulated life sciences, the audit trail has a specific legal content. In the United States, 21 CFR 11.10(e) requires persons using closed systems for electronic records to use "secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records", adds that "record changes shall not obscure previously recorded information", and requires the audit trail to be retained at least as long as the records it covers and to be available for FDA review and copying ^[7].

In the EU, Annex 11 to the GMP guidelines (computerised systems), clause 9, states that "consideration should be given, based on a risk assessment, to building into the system the creation of a record of all GMP-relevant changes and deletions (a system generated 'audit trail')", that for a change or deletion of GMP-relevant data "the reason should be documented", and that audit trails "need to be available and convertible to a generally intelligible form and regularly reviewed" ^[8]. The Commission's EudraLex Volume 4 page lists this January 2011 revision as the current Annex 11 ^[9]. A revised Annex 11 went to stakeholder consultation from 7 July to 7 October 2025 and had not been adopted when this article was checked on 28 October 2026 ^[10].

Outside life sciences, NIST SP 800-53 Rev. 5 control AU-3 asks that audit records establish what type of event occurred, when and where it occurred, its source and outcome, and the identity of individuals or entities associated with it ^[11].

For Halden, the audit trail on the field shows:

This proves that the change happened, who made it, when, and that the earlier value is still visible, as 11.10(e) requires. It does not prove the new value is right. "New certificate received" is a reason, not evidence: the audit trail does not contain the certificate, or the passage that supports 2027-05-31.

See also the glossary entry for audit trail.

Where each fails alone

The three records fail in different ways. Each failure below is a realistic outcome for Halden's dashboard.

The pattern is consistent. Lineage localizes impact across systems. Provenance justifies a value. The audit trail proves the history of actions on a record. None of the three substitutes for another.

Combining them into one chain of custody

Chain of custody, a term from evidence handling, is the documented sequence of who held an item and what was done to it from collection to presentation (glossary: chain of custody). For a data value, a usable chain of custody needs all three records, linked by stable identifiers:

1. Value-level provenance as the anchor: the value, its source document, the supporting passage, the activity that extracted or computed it, and the responsible agent.
2. Audit trail entries that reference the same value or record identifier, so each change can be matched to the provenance of the new value. Annex 11's "reason should be documented" becomes much stronger when the reason is a pointer to a provenance record rather than free text.
3. Lineage that carries the value's identifier, or at least its record identifier, through downstream jobs, so that a value displayed on a dashboard can be traced back to the provenance record that justifies it.

In the Halden example, a combined record would let the inspector start from the dashboard tile, follow lineage to the supplier_master row, read the audit trail entry of 2026-02-10, and open the provenance record that points to page 1 of the certificate with the quoted text. Each hop uses a different system, but the identifiers connect them.

Two design points make the combination work in practice. First, provenance must be versioned with the value: when the audit trail records a change, the new value needs its own provenance record; overwriting the old one destroys the history. Second, identifiers must be stable across systems: a lineage graph keyed on table names and an audit trail keyed on row IDs cannot be joined to a provenance store keyed on something else.

Regulatory pull: which rule asks for which

Regulations rarely use the words "lineage" and "provenance", but their requirements map onto them. The table cites only provisions quoted above or in the linked sources; it is a reading aid, not legal advice.

Sources for the table: 21 CFR 11.10; Annex 11; NIST SP 800-53 Rev. 5; BCBS 239, paragraphs 36, 37 and 39; Regulation (EU) 2024/1689, Article 10. The AI Act's high-risk requirements apply from 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems, as amended by Regulation (EU) 2026/1744.

Life sciences rules pull hardest on audit trails; banking supervision pulls on lineage and reconciliation; AI regulation pulls on the origin of data. An organization subject to more than one needs all three records and the identifiers that connect them.

Comparison table

Summary

Lineage, provenance and audit trail are three answers to three questions. A catalog's lineage graph tells you which pipelines to check. A provenance record tells you why a value is believed. An audit trail tells you who touched it. A value is defensible only when the three are kept together and joined by identifiers that do not change from one system to the next.

A system of record that keeps a cited, dated source behind every value and preserves every prior value is one way to hold all three together; that is the property Reliable Context is designed around.

Sources

1. W3C, PROV-DM: The PROV Data Model (Recommendation, 30 April 2013): definition of provenance; entity, activity, agent; derivation and attribution.
2. OpenLineage, Object model: definitions of Job, Run, Dataset and dataset naming.
3. OpenLineage, Facets and extensibility: definition of a facet.
4. NIST CSRC glossary, "lineage": lineage definition (from the NIST Privacy Framework).
5. NIST CSRC glossary, "provenance": provenance definition (from SP 800-53 Rev. 5 and others).
6. NIST CSRC glossary, "audit trail": audit trail definition.
7. 21 CFR 11.10, Controls for closed systems (eCFR): paragraph (e) audit trail requirement.
8. EudraLex Volume 4, Annex 11: Computerised Systems (revision January 2011): clause 9, audit trails.
9. European Commission, EudraLex Volume 4 index: lists the January 2011 revision of Annex 11.
10. European Commission, Stakeholders' consultation on EudraLex Volume 4: Chapter 4, Annex 11 and new Annex 22: consultation 7 July to 7 October 2025.
11. NIST SP 800-53 Rev. 5, Security and Privacy Controls: control AU-3, content of audit records.
12. Basel Committee, Principles for effective risk data aggregation and risk reporting (BCBS 239), January 2013: Principle 3, paragraphs 36, 37 and 39.
13. Regulation (EU) 2024/1689 (AI Act): Article 10(2)(b).
14. Regulation (EU) 2026/1744 (Digital Omnibus on AI): application dates for high-risk requirements.