Zero-Shot Generalization from Pre-trained Instruction Knowledge
The ability to understand instructions, acquired during the pre-training phase as part of general language comprehension, can enable a model to perform zero-shot learning. This means the model can generalize and successfully execute tasks based on instructions it has never seen before.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
A researcher pre-trains a new language model on a vast and diverse dataset of text from the internet. Without any subsequent specialized training, the researcher tests the model with the prompt: 'Summarize the following paragraph in one sentence: [paragraph text]'. The model successfully produces a coherent, one-sentence summary. Which of the following statements provides the most accurate explanation for this capability?
Zero-Shot Generalization from Pre-trained Instruction Knowledge
A large language model's capacity to understand and execute a wide range of tasks based on textual prompts is primarily instilled through a specialized training stage that is separate from and follows its initial, general-purpose language learning phase.
Impact of Pre-training Data on Instruction Following
Learn After
A large language model, trained on a vast and diverse corpus of internet text, is given the following novel instruction: 'Analyze the sentiment of the following customer review and express it as a single emoji.' The model has never been explicitly trained on this specific 'text-to-emoji' task. Despite this, it correctly outputs a '😠' for a negative review. Which statement best explains the model's ability to successfully complete this new task?
Evaluating a Development Strategy for a New AI Feature
Evaluating Zero-Shot Generalization in a High-Stakes Application