These are the GLUE Test results after fine-tuning:

• Both BERT$_{BASE}$ and BERT$_{LARGE}$ outperform all systems on all tasks by a substantial margin, obtaining 4.5% and 7.0% respective average accuracy improvement
• BERT$_{LARGE}$ significantly outperforms BERT$_{BASE}$ across all tasks, especially those with very little training data