**SP+** is the paper's notational extension of strict parity. Where strict parity (SP) fixes encoder, candidate pool size $$m$$, cutoff $$k$$, split, and exact-ID matching rule across compared systems, **SP+ additionally allows a learned reranker** on top of the shared dense candidate pool. SP+ results are reported in a separate column from SP so that any reranker-driven gain is not silently folded into a strict-parity headline number. SP+ is meaningful only relative to the underlying SP contract, so the SP definition is a direct prerequisite.

SP+ Extension: Strict Parity Plus a Learned Reranker, Reported Separately

Two leakage-controlled splits for prerequisite-QA evaluation, stated as plain definitions at first use:

- A **question-disjoint split** removes any test instance whose question string (or near-duplicate) appears in training. This blocks gains that come from memorizing question text rather than reasoning over the prerequisite graph.
- A **target-concept-disjoint split** is strictly stronger: in addition to removing question-string overlap, it removes test items whose **gold target concept IDs** appear as training supervision. This blocks gains that come from having seen the held-out target concept during training.

The two splits are the concrete instruments by which the leakage audit's two overlap kinds (question-string and target-concept) are operationalized; they are used together in headline retrieval comparisons so that surviving gains can be attributed to the retrieval policy rather than to overlap with training material.

Question-Disjoint and Target-Concept-Disjoint Splits in Prerequisite-QA Leakage Control

**Strict parity (SP)** is the evaluation contract used in this paper to compare graph-aware RAG systems on a like-for-like basis. Under SP, the following are **fixed across all compared systems** in the headline comparisons: the encoder, the dense candidate pool size, the top-$$k$$ cutoff, the data split, and the matching rule that decides when a retrieved item counts as correct. By fixing these five interface choices, any remaining performance difference between systems can be attributed to the graph/retrieval policy itself rather than to drift in the retrieval interface. Token-cap effects are analyzed separately under a shared serialization policy and are not part of the SP headline contract.

Strict Parity (SP): Matched-Interface Evaluation Contract for Graph-Aware RAG Comparisons

A **token-cap diagnostic** is a separate evaluation in which two retrieval systems are compared after a **shared token-aware serialization policy** is applied to their returned context. Because the policy that serializes candidates into the LLM context (ordering, truncation, length-awareness) is held identical across the compared systems, any remaining difference reflects the ranked retrieval output rather than serialization choices. In this paper the token-cap diagnostic is reported separately from the strict-parity (SP) headline retrieval contract: SP fixes encoder, candidate pool, cutoff, split, and matching rule; the token-cap diagnostic adds a matched serialization layer on top so that tight-budget behavior can be inspected without conflating it with the headline retrieval comparison.

Token-Cap Diagnostic: Shared Token-Aware Serialization Comparison

This organizer groups method, retrieval, ranking, and graph-structure details used by Introduction.

Google

This organizer groups the paper content from Introduction.

Introduction in Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls

Root URL: https://1iman.com/
Canonical paper URL: https://1iman.com/assets/working-papers/pdf/emnlp-auditable-strict-parity-evaluation-of-prerequisite-graph-retrieval-for-rag-under-leakage-controls.pdf
Portfolio PDF URL: https://1iman.com/assets/working-papers/pdf/emnlp-auditable-strict-parity-evaluation-of-prerequisite-graph-retrieval-for-rag-under-leakage-controls.pdf
Paper title: Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls
Use this Reference node when citing paper-derived 1Cademy proposals from the Researcher agent.

Reference: Auditable Strict-Parity Evaluation of Prerequisite-Graph Retrieval for RAG under Leakage Controls

Method Details for Introduction (Auditable Strict-Parity Graph-RAG Paper)

This organizer groups metrics, datasets, statistical protocol, and result evidence used by Introduction.

Evaluation Evidence for Introduction (Auditable Strict-Parity Graph-RAG Paper)

This organizer groups scope, caveats, licensing, ethical, and interpretation notes connected to Introduction.

Learn Before

Related

Learn After