These are the SQuAD v1.1 Test results after fine-tuning:

- The researchers’ best performing system outperforms the top leaderboard system by +1.5 F1 in ensembling and +1.3 F1 as a single system
- The researchers’ single BERT model outperforms the top ensemble system in terms of F1 score


SQuAD v1.1 Results

The SQuAD v1.1 is a collection of 100k crowdsourced question/answer pairs.

Ohio State University, Columbus

This paper presents BERT fine-tuning results on 11 Natural Language Processing (NLP) tasks.

BERT Experiments

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

https://paperswithcode.com/method/bert

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

The GLUE benchmark is a collection of diverse natural language understanding tasks.

General Language Understanding Evaluation (GLUE)

Stanford Question Answering Dataset 
(SQuAD v1.1)

The SQuAD 2.0 task extends the SQuAD 1.1 problem definition by allowing for the possibility that no short answer exists in the provided paragraph, making the problem more realistic.

Stanford Question Answering Dataset 
(SQuAD v2.0) 

The SWAG dataset contains 113k sentence-pair completion examples that evaluate grounded common-sense inference.

Situations With Adversarial Generations (SWAG)


Ablation studies on BERT are attempts to understand the relative importance of the various pieces of BERT’s functionality by testing them in isolation.

Learn Before

Related

Learn After