Learn Before
Dataset

HotpotQA Multi-Hop QA Benchmark

HotpotQA is a large-scale multi-hop question answering dataset introduced by Yang et al. at EMNLP 2018. It contains approximately 113{,}000 English question-answer pairs built over Wikipedia, where each question is constructed to require reasoning over two or more supporting paragraphs rather than a single passage. Every question is annotated at the sentence level with supporting facts that justify the answer, enabling joint evaluation of answer correctness and evidence selection. The benchmark defines two standard settings: a distractor setting, in which a system must answer each question given 1010 paragraphs (22 gold plus 88 lexically related distractors), and a fullwiki open-domain setting, in which the supporting paragraphs must be retrieved from the entire Wikipedia corpus. The release also includes comparison questions that contrast attributes of two entities. Standard metrics are Exact Match and F1 for both the predicted answer and the predicted supporting-fact set, with retrieval components typically scored by Recall@kk over the fullwiki candidate pool.

0

1

Updated 2026-05-18

Contributors are:

Who are from:

Tags

Science

Related