We need to select questions that meet a certain level of difficulty from the SQuAD database to accurately assess reading comprehension from difficult concepts. These criteria include:

- Syntactic variation: word structure of a sentence in the text is different from how it is phrased in the question (eg. Question: What Shakespeare scholar is currently on the faculty? Phrase from the text: Current faculty include...)
- Multiple sentence reasoning: multiple sentences required from text to answer a question, not just a single sentence

The SQuAD paper analyzes other question types, but for purposes of our research we only need to meet these criteria

Criteria to Consider For Question Extraction

Examples of reading comprehension questions derived from the Wikipedia article on economic inequality include:

- What impact does higher worker productivity and leveled pay have on lower earners?
- What happens when business overpays their workers?
- How do competing businesses repel workers?
- What impact did the high school education movement have on the wages of unskilled workers?
- What do strong labor movement correlate with?

Example Reading Comprehension Questions on Economic Inequality

Reading comprehension questions and their corresponding Wikipedia articles can be sourced from the Stanford Question Answering Dataset (SQuAD) versions 1.1 and 2.0. These databases provide large-scale collections of crowdsourced question-answer pairs derived from Wikipedia text, serving as valuable resources for reading comprehension research.

Boston University

Google

Knowledge (expertise, skills, and information) being exchanged on the platform Wikipedia, an open collaborative online encyclopedia 

Knowledge sharing on Wikipedia

The SQuAD v1.1 is a collection of 100k crowdsourced question/answer pairs.

Stanford Question Answering Dataset 
(SQuAD v1.1)

The SQuAD 2.0 task extends the SQuAD 1.1 problem definition by allowing for the possibility that no short answer exists in the provided paragraph, making the problem more realistic.

Stanford Question Answering Dataset 
(SQuAD v2.0) 

Citations or references are used to identify the sources of the information found in Wikipedia. They help to improve Wikipedia's credibility and allow users to verify the information found and to further find information if needed. 

Citations of References on Wikipedia

One field of study that Wikipedia articles focus on is health. Illnesses/diseases, treatments, preventative care, and most recently COVID-19 are all examples of health-related information one can find on Wikipedia. 

Health-related information on Wikipedia

Place your references about knowledge sharing on collaborative platforms here

CSCW 2021 References

Reading Comprehension Datasets from Wikipedia Articles

These are the SQuAD v1.1 Test results after fine-tuning:

- The researchers’ best performing system outperforms the top leaderboard system by +1.5 F1 in ensembling and +1.3 F1 as a single system
- The researchers’ single BERT model outperforms the top ensemble system in terms of F1 score


SQuAD v1.1 Results

These are the SQuAD v2.0 Test results after fine-tuning:
- There is a +5.1 F1 improvement over the previous best system

Learn Before

Related

Learn After