rag evaluation
8 Articles
RAG Query Rewriting: Does It Preserve the Question?
Evaluation protocol for RAG query rewriting in 2026: intent contracts, paired qualifier fixtures, constraint preservation vs regression, McNemar tests, and release, fallback or clarify rules.
LLM conflicting sources: a counterfactual fusion test
A 2026 evidence review of LLM source conflict resolution, reconstructing tabular-fusion F1 results and specifying entity masking, source-dependence tests, abstention metrics and reproducible provenance.
Conformal RAG Guarantees: A Loss-and-Denominator Audit
A 2026 audit of conformal RAG guarantees: compare set coverage, bounded loss and selective error, inspect SAFER's finite budget, and reproduce a scoped binomial test with synthetic counterexamples.
Multi-Hop RAG Citation Completeness: An Evidence-Set Test
A 2026 evidence-set test for multi-hop RAG citation completeness: distinguish answer correctness, joint support and missing hops, preserve source versions, and audit alternative evidence routes.
RAG evaluation dataset leakage audit as sources change
A 2026 audit guide for RAG evaluation sets: trace question and source lineage, test answer and training exposure, compare fixed with rolling corpora, and report paired score uncertainty.
LLM Context Compression Evidence Loss: An Acceptance Test
A 2026 guide to testing LLM context compression for evidence loss, with paired source bundles, exact-span scoring, citation-grounding data, and release rules for defensible answers.
LLM Judge Groundedness Evaluation Calibration Protocol
A 2026 protocol for calibrating an LLM groundedness judge with controlled contradictions, paired judge-by-generator tests, confidence intervals, human review, and release gates.
RAG Fault Isolation: Is the Answer Defensible?
A 2026 fault-isolation protocol for RAG answers, covering six observable boundaries, claim-level metrics, controlled evidence tests, source currency, and paired evaluation.