Question-Disjoint and Target-Concept-Disjoint Leakage Controls
Question-disjoint and target-concept-disjoint controls are the two leakage-auditing splits used in this paper for canonical prerequisite-QA benchmarks. The audit is motivated by the observation that canonical splits reuse many exact question strings and target concepts across train and test, so a system can match memorized surface forms rather than retrieve true prerequisites. Under the question-disjoint control, no test question string appears in training. Under the target-concept-disjoint control, no test target concept appears as a target in training. Results obtained only under these controls are treated as the trustworthy evidence, and the paper restricts its remaining conclusions to curated, template-based prerequisite QA.
0
1
Tags
Science
Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls
Learn After
Method Details for Question-Disjoint and Target-Concept-Disjoint Leakage Controls (Auditable Strict-Parity Graph-RAG Paper)
Evaluation Evidence for Question-Disjoint and Target-Concept-Disjoint Leakage Controls (Auditable Strict-Parity Graph-RAG Paper)
Scope Notes for Question-Disjoint and Target-Concept-Disjoint Leakage Controls (Auditable Strict-Parity Graph-RAG Paper)