MRR is designed for systems that return a short ranked list of answers or passages for each test set question, which we can compare against the (human-labeled) correct answer. MRR is computed by the formula below:

Q = test set of questions

rank = rank of the first correct answer

MRR = sum of 1 / rank for all the questions in Q / size of Q

Mean Reciprocal Rank (MRR)

Exact match is one of the two metrics used to evaluate reading comprehension systems. 

Exact Match = The % of predicted answers that match the gold answer exactly.

Exact Match

F1 score is one of the metrics used to evaluate reading comprehension systems.
It takes the average word/token overlap between predicted and gold answers. Treat the prediction and gold as a bag of tokens, and compute F1 for each question, then return the average F1 over all questions.

F1 score

Common approach(es) to evaluating Factoid question answering and reading comprehension systems.

Hamilton College

Factoid questions are questions that can be answered with simple facts expressed in short texts, which normally start with words including but not limited to 'when', 'what', and 'where'.

Factoid Questions

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Learn Before

Related

Learn After