A question-answering model is given a query and a context passage. It processes the combined text and generates a final contextualized embedding for every token. To identify the specific text span within the passage that answers the query, the model must calculate start and end probabilities for each potential token. Which set of embeddings should be used as input to the prediction networks that perform this calculation?

Debugging a Span Prediction Model

In a span prediction model designed for question answering, after the entire input (query + context) has been processed to generate contextualized token embeddings, the prediction networks for the answer's start and end positions must evaluate the embeddings for all tokens in the original input sequence.