Glossary
Point-in-time reproducibility
Point-in-time reproducibility is the ability to reconstruct a past decision's inputs and recorded context.
Time and versioning
Point-in-time reproducibility means a past result can be reconstructed from the information, rules, and source versions available at a declared point. It is stronger than showing an old displayed value. A decision may depend on several linked records, a classification, a source excerpt, and the treatment of missing information. If one of those dependencies changes without a retained version, a later replay may produce a different answer while appearing historical.
A reproducible record identifies the decision time, the evidence available then, the exact references used, and the applicable interpretation or decision rules. A replay should distinguish a faithful reconstruction from a new assessment of the old date using evidence found later. Time zones, identity changes, and corrected source documents often expose gaps. Retained source copies and versioned lineage give a reviewer something concrete to inspect beyond a dated screenshot.
This property lets an organization explain why it acted on a value even if that value was later corrected. It also makes it possible to compare a past decision with a current reassessment without collapsing the two. As-of queries retrieve temporal states; point-in-time reproducibility covers the complete chain needed to rederive an outcome. An audit trail records activity, but activity alone does not preserve every input required for reconstruction.
Example
Halden Therapeutics AG (fictional) revisits a 1 July eligibility decision using the source copy, identity link, rule version, and recorded knowledge state available on 1 July.
What has to be preserved
To reproduce a past decision one needs four things: the data as they were, the code or rules that were applied, the configuration and parameters, and the environment in which the computation ran. The National Academies report on reproducibility and replicability in science defines reproducibility as obtaining consistent results using the same input data, computational steps, methods and code, and conditions of analysis, and its recommendations stress the need for transparency about all of these. The paper Ten Simple Rules for Reproducible Computational Research gives practical guidance, among them to record how every result was produced, to version custom scripts, to archive the exact versions of programs, and to provide public access to data and scripts.
For business decisions, a fifth item is added: the sources consulted. A figure that rested on a web page cannot be reproduced if the page changes, so the page must be retained, as described under retained source copy. Each item needs a version identifier, and the decision record needs to reference them all together.
Mechanisms
Several mechanisms together provide the capability. Bitemporal or versioned storage gives as-of queries over the data; lakehouse formats such as Delta Lake and Apache Iceberg let a job pin a table to a version. Version control and tagged releases pin the code. Data lineage records which inputs and which code produced each output. The W3C PROV vocabulary can describe the whole chain in an exchangeable form, and the article on W3C PROV for LLM context applies it to the material given to a language model, where the prompt, the retrieved passages and the model version must all be recorded.
Reproducibility has limits that should be documented. Non-deterministic components, such as sampling in a language model or an unordered parallel computation, may give different outputs from the same inputs; in that case the aim becomes reproducing the inputs and the recorded output, and testing that a rerun is statistically consistent. Third-party services change without notice, so the record of what they returned matters more than the ability to call them again.
A worked example
In March a committee approved a supplier after a risk score of 0.62 for Halden Therapeutics AG's (fictional) contract manufacturer. In September an auditor asks how the score was derived. The decision record references the model version, the rule set version, the table snapshot used for the inputs, and the retained copies of three external reports. The team recreates the environment from the tagged release, runs the scoring against the pinned snapshot and reproduces 0.62. A second run using current data gives 0.71, because the manufacturer's inspection record was updated in June; the difference is explained by the data and not by the method. The audit question is answered with evidence and not recollection.
Checks a reviewer can run
Pick a past decision and attempt to reproduce its input. Ask for the data version, code version, configuration and sources, and time how long it takes to assemble them. Rerun and compare. Record the reasons for any difference. Ask what proportion of decisions carry a complete record, and what is done about those that do not.
Related terms and boundaries
Reproducibility draws on bitemporal data, data lineage, the audit trail and retained source copies, and is limited by the retention schedule, which decides how long the pieces survive. It is the property that makes a system of record defensible after the fact.
Common misreadings
- Calling an old screenshot a reproducible decision.
- Applying current rules to past evidence without disclosure.
- Preserving outputs but discarding their source versions.
Appears in
- Snapshots vs bitemporal history
Snapshots preserve states at selected moments; bitemporal history distinguishes when a fact applied from when it was recorded.
- Detecting source drift
Compare current source content with the retained cited version and reassess changed support.
- Reproducing a past state
Reconstruct what was known at a specified instant using observation time, effective time and earlier decisions.
References
- W3C, PROV-O: The PROV Ontology (2013).
- National Academies of Sciences, Engineering, and Medicine (2019), Reproducibility and Replicability in Science.
- Sandve et al. (2013), Ten Simple Rules for Reproducible Computational Research, PLOS Computational Biology.
- Delta Lake documentation, Table batch reads and writes (time travel).
- Apache Iceberg documentation, Queries (time travel).
- W3C (2013), PROV-O: The PROV Ontology.
Found an error on this page? See corrections for what counts as one, how to report it and the dated log.
Enquiry
Where does "Point-in-time reproducibility" bear on a decision of yours?
This entry sits under time and versioning. If a decision in your organization turns on it, send the entity concerned, the sources you fuse to describe it and who may read which of them. We reply with what a first record would cover.
Or write to [email protected] in your own words.