On **MOOC-CS**, the paired-bootstrap $$\Delta\text{R@}k$$ curve between **Adaptive (heuristic) depth gating** and the **fixed-depth hierarchical baseline** **stays near zero throughout** $$k$$, in contrast to LectureBank-Full where adaptive shows a pointwise advantage at $$k=4$$. Even where the LectureBank-Full pointwise effect appears, it is weaker than the diffusion-and-quota gain. This reading supports the paper's descriptive conclusion that adaptive depth gating's tight-budget benefit is **dataset-specific and not regime-wide**.

MOOC-CS ΔR@k Curve Stays Near Zero Across k (Adaptive vs Hierarchical)

On **MOOC-CS**, swapping the default English query encoder for a **multilingual encoder** combined with a **CJK-only query rewrite** is treated as a **control experiment**, not as a headline strict-parity comparison. Changing the encoder and the query interface breaks the strict-parity invariant that the encoder and query interface are held constant across compared systems, so any resulting retrieval gains cannot be attributed to the graph component alone. The configuration is reported as a separate language-matched diagnostic on MOOC-CS that informs interpretation of MOOC-CS results without entering the headline numbers.

Multilingual Encoder Plus CJK-Only Query Rewrite as a Non-Headline Control on MOOC-CS

Under this paper's strictest leakage control on prerequisite QA splits, only a small number of **unique held-out target concepts** remain after removing both train/test question-string reuse and target-concept overlap: **21 unique held-out targets on LectureBank-Full** and **18 on MOOC-CS**, distributed within just two question families. Because the surviving evidence is this narrow, the paper bounds its leakage-controlled headline conclusions to **curated, template-based prerequisite QA** rather than projecting them to open-domain retrieval.

Bounded Held-Out Targets After Strictest Leakage Control (21 LectureBank-Full, 18 MOOC-CS)

This organizer groups scope, caveats, licensing, ethical, and interpretation notes connected to MOOC-CS Prerequisite Benchmark.

Google

**MOOC-CS** is a computer-science-focused prerequisite-relation benchmark assembled from Massive Open Online Course (MOOC) materials, originating from Pan et al. (ACL 2017). It comprises Coursera lecture captions in the **Data Structures and Algorithms** and **Machine Learning** domains, with course core concepts extracted from the captions and manually annotated ordered concept pairs labeled as prerequisite or not. Each course release provides three file types: a JSON Captions file (tokenized, POS-tagged video transcripts), a Core_concepts file listing per-domain concepts, and a Labeled file storing concept pairs with their prerequisite labels. The benchmark is used to evaluate prerequisite-link retrieval systems, with a fixed candidate concept pool and ranking metrics such as Recall@$$k$$. The default release is English-only, which is why graph-aware retrieval studies report separately on language-matched query and encoder controls when extending evaluations to multilingual MOOC settings.

MOOC-CS Prerequisite Benchmark

Root URL: https://1iman.com/
Canonical paper URL: https://1iman.com/assets/working-papers/pdf/emnlp-auditable-strict-parity-evaluation-of-prerequisite-graph-retrieval-for-rag-under-leakage-controls.pdf
Portfolio PDF URL: https://1iman.com/assets/working-papers/pdf/emnlp-auditable-strict-parity-evaluation-of-prerequisite-graph-retrieval-for-rag-under-leakage-controls.pdf
Paper title: Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls
Use this Reference node when citing paper-derived 1Cademy proposals from the Researcher agent.

Reference: Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls

This organizer groups method, retrieval, ranking, and graph-structure details used by MOOC-CS Prerequisite Benchmark.

Method Details for MOOC-CS Prerequisite Benchmark (Auditable Strict-Parity Graph-RAG Paper)

This organizer groups metrics, datasets, statistical protocol, and result evidence used by MOOC-CS Prerequisite Benchmark.

Learn Before

Related

Learn After