Learn Before
Qualitative Manual Bundles Remain Non-Evidentiary Until Multi-Annotator Replication
The paper's released qualitative bundles are useful for future audits but are explicitly non-evidentiary: they cannot be used to support inferential claims until they are replicated with multiple independent annotators. Treating the bundles as audit material rather than evidence is what allows the paper to release them without using them to validate automatic judges or to argue for end-to-end QA improvements.
0
1
Tags
Science
Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls
Related
Generation as Context-Quality Diagnostic, Not a Headline Claim
Claims Explicitly Avoided: Auto-Judge Validation, Semantic-Evasion Effects, End-to-End QA Gains
Qualitative Manual Bundles Remain Non-Evidentiary Until Multi-Annotator Replication
Analysis-Section Scope Statement: Evidence Specific to Curated, Template-Based Prerequisite QA
HotpotQA External-Validity Probe: Adaptive Depth Does Not Transfer to a Denser Non-Prerequisite Graph (FullWiki-1k: Flat 93.4 / Hier 92.9 / Adaptive 94.0 R@10)