Learn Before
Acquiring Instruction Knowledge During Pre-training
Because task instructions are provided as text and integrated into the model's input, the general ability to comprehend instructions can be learned during the pre-training phase as an extension of the model's overall language understanding capabilities.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.2 Generative Models - Foundations of Large Language Models
Related
Acquiring Instruction Knowledge During Pre-training
A developer is using a pre-trained language model for a new task: converting a user's informal description of a meeting into a structured JSON object with 'title', 'date', and 'attendees' keys. Which of the following textual instructions, provided to the model along with the user's description, would be most effective at consistently producing the correct output format?
Enabling Instruction Following via Pre-training
Choosing Appropriate Instruction Formats
Diagnosing and Refining Task Instructions
Universal Language Framework via Textual Inputs
Learn After
A researcher pre-trains a new language model on a vast and diverse dataset of text from the internet. Without any subsequent specialized training, the researcher tests the model with the prompt: 'Summarize the following paragraph in one sentence: [paragraph text]'. The model successfully produces a coherent, one-sentence summary. Which of the following statements provides the most accurate explanation for this capability?
Zero-Shot Generalization from Pre-trained Instruction Knowledge
A large language model's capacity to understand and execute a wide range of tasks based on textual prompts is primarily instilled through a specialized training stage that is separate from and follows its initial, general-purpose language learning phase.
Impact of Pre-training Data on Instruction Following