Learn Before
Enabling Zero-Shot Learning through Instruction Understanding
A model's ability to understand instructions, an emergent capability from pre-training, is a key factor that enables zero-shot learning. This allows the model to generalize its knowledge to perform new tasks for which it has not been explicitly trained, simply by following novel textual instructions it has never encountered before.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Enabling Zero-Shot Learning through Instruction Understanding
Computational Expense of Training LLMs from Scratch
Difficulty in Collecting Labeled Data for Instruction Pre-training
A research lab develops a new large language model by training it on a massive dataset consisting solely of digitized books and encyclopedias. The model becomes exceptionally proficient at generating coherent, factual paragraphs. However, when users give it a direct command, such as "Translate 'hello' into French," the model often responds with a continuation like "is a common English greeting," instead of "Bonjour."
Which of the following best analyzes the most likely reason for this specific failure?
Pre-training Data Strategy for a Command-Following Model
Pre-training a Specialized Code Assistant
Learn After
A large language model, pre-trained on a diverse corpus of internet text, is given the following prompt for the first time: "Rephrase the following sentence in the style of a 1920s radio announcer: 'The new smartphone will be available next week.'" The model successfully responds with something like: "Hark, good citizens! The magnificent new pocket telephone contraption shall grace our stores in but a week's time!" Which statement best analyzes the primary reason the model can successfully complete this novel task?
LLM Feature Development Strategy
Evaluating the Source of Zero-Shot Capabilities