Bounded Held-Out Targets After Strictest Leakage Control (21 LectureBank-Full, 18 MOOC-CS)
Under this paper's strictest leakage control on prerequisite QA splits, only a small number of unique held-out target concepts remain after removing both train/test question-string reuse and target-concept overlap: 21 unique held-out targets on LectureBank-Full and 18 on MOOC-CS, distributed within just two question families. Because the surviving evidence is this narrow, the paper bounds its leakage-controlled headline conclusions to curated, template-based prerequisite QA rather than projecting them to open-domain retrieval.
0
1
Tags
Science
Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls
Related
LectureBank-Full R@10 Gain from Diffusion and Role-Aware Quotas
LectureBank-Full Configuration Used in Hierarchical Prerequisite RAG (208 Concepts, 899 Edges, 1,421 QA)
LectureBank-Full Target-Disjoint R@10 Result (n=164): Diffusion Gain Survives, Adaptive Tied
LectureBank-Full Generation Diagnostic: Token-F1 1.9 → 18.3, EM Stays 0.0
LectureBank-Full Error Taxonomy: Residual Misses Are Near-Misses Along the Local Prerequisite Graph
LectureBank-Full Paired Delta: Adaptive vs Hierarchical Baseline = +0.7 [-2.1, +3.6]
Token-Cap Comparison on LectureBank-Full: Adaptive Loses More as Cap Tightens
LectureBank-Full Tight-Budget Advantage of Adaptive Depth Gating (Mean ΔR@k = +2.13 over k∈{1,2,3,4})
LectureBank-Full ΔR@k Peaks at k=4 (+6.4 Points, CI [1.0, 11.7])
LectureBank-Full Diffusion Gain over Static Parent Expansion (~18 R@10 Points)
Bounded Held-Out Targets After Strictest Leakage Control (21 LectureBank-Full, 18 MOOC-CS)
LectureBank-Full Decomposition: Diffusion+Quotas Drive ~18 R@10 Points; Contrast Gating Adds At Most ~1 Point (Statistically Tied)
Bounded Held-Out Targets After Strictest Leakage Control (21 LectureBank-Full, 18 MOOC-CS)
Corrective Field Lesson: Four Required Ingredients for Honest Graph-RAG Claims
MOOC-CS ΔR@k Curve Stays Near Zero Across k (Adaptive vs Hierarchical)
Multilingual Encoder Plus CJK-Only Query Rewrite as a Non-Headline Control on MOOC-CS
Bounded Held-Out Targets After Strictest Leakage Control (21 LectureBank-Full, 18 MOOC-CS)