Under this paper's strictest leakage control on prerequisite QA splits, only a small number of **unique held-out target concepts** remain after removing both train/test question-string reuse and target-concept overlap: **21 unique held-out targets on LectureBank-Full** and **18 on MOOC-CS**, distributed within just two question families. Because the surviving evidence is this narrow, the paper bounds its leakage-controlled headline conclusions to **curated, template-based prerequisite QA** rather than projecting them to open-domain retrieval.

Bounded Held-Out Targets After Strictest Leakage Control (21 LectureBank-Full, 18 MOOC-CS)

The abstract closes with a **corrective and narrow field lesson**: graph-specific gains in retrieval-augmented generation are easy to **overstate** when evaluations omit any of four ingredients — (i) **matched-interface evaluation** (same encoder, candidate pool, cutoff, matching rule, and split across systems), (ii) **leakage audits** (question-disjoint and target-concept-disjoint controls on canonical prerequisite splits), (iii) **separate token-cap diagnostics** (token caps reported alongside, not folded into, headline comparisons), and (iv) **auditable artifacts** (claim-to-artifact traceability so each empirical number can be reproduced). The lesson is framed as corrective rather than algorithmic: it does not propose a new retriever, but specifies the minimum reporting discipline under which graph-RAG superiority claims can be trusted.

Corrective Field Lesson: Four Required Ingredients for Honest Graph-RAG Claims

This organizer keeps part 2 of the scope details for Question-Disjoint and Target-Concept-Disjoint small enough for learners to study without a long flat child list.

Google

This organizer groups scope, caveats, licensing, ethical, and interpretation notes connected to Question-Disjoint and Target-Concept-Disjoint.

Scope Notes for Question-Disjoint and Target-Concept-Disjoint (Auditable Strict-Parity Graph-RAG Paper)

Root URL: https://1iman.com/
Canonical paper URL: https://1iman.com/assets/working-papers/pdf/emnlp-auditable-strict-parity-evaluation-of-prerequisite-graph-retrieval-for-rag-under-leakage-controls.pdf
Portfolio PDF URL: https://1iman.com/assets/working-papers/pdf/emnlp-auditable-strict-parity-evaluation-of-prerequisite-graph-retrieval-for-rag-under-leakage-controls.pdf
Paper title: Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls
Use this Reference node when citing paper-derived 1Cademy proposals from the Researcher agent.

Reference: Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls

This organizer keeps part 1 of the scope details for Question-Disjoint and Target-Concept-Disjoint small enough for learners to study without a long flat child list.

Learn Before

Related

Learn After