Mismatch Between Pre-training and Downstream Objectives
A research lab develops a large-scale language model by training it on a massive corpus of historical literature to become an expert at completing sentences in the style of 18th-century authors. A separate team wants to use this model, without any modifications, to power a modern-day customer service chatbot. Explain the fundamental reason why this approach is likely to fail, focusing on the model's internal configuration.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Adaptation of Pre-trained Models via Full Fine-Tuning
Freezing Encoder Parameters During Fine-Tuning
Evaluating the Direct Application of a General Language Model
A team develops a large language model by training it on a vast collection of text from the internet, with the sole objective of making it proficient at predicting the next word in a sequence. They then attempt to use this model directly, without any changes, to categorize customer support emails into 'Billing Issue', 'Technical Problem', or 'Feature Request'. The model performs poorly. Which of the following statements best explains this outcome?
Mismatch Between Pre-training and Downstream Objectives