Tied-at-R@10 Reading is Non-Rejection of Equivalence Under a Power Bound, Not a Positive Equivalence Claim
The Conclusion explicitly frames the headline tied-at-R@10 finding between adaptive and fixed-depth hierarchical retrieval as a non-rejection of equivalence under the prevailing statistical power bound, not a positive claim that the two policies are equivalent. The power bound is set by the strictest leakage controls, which expose only 21 unique held-out targets on LectureBank-Full and 18 on MOOC-CS across two question families. With this few unique targets, paired-bootstrap intervals are wide and a true small advantage for either policy could not be detected; the correct reading is therefore that the data do not reject equivalence at R@10, while leaving open both directions of a small effect.
0
1
Tags
Science
Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls
Related
Defensible Narrow Conclusion: Graph Diffusion Helps on Curated Template-Based Prerequisite QA Only Under Aligned, Controlled First-Stage Retrieval
Tied-at-R@10 Reading is Non-Rejection of Equivalence Under a Power Bound, Not a Positive Equivalence Claim
Deterministic Diffusion Serves as a Controlled Test Case, Not a Broad New-Retriever Claim
Tied-at-R@10 Reading is Non-Rejection of Equivalence Under a Power Bound, Not a Positive Equivalence Claim