Learn Before
Limitation of Pre-trained LLMs: Next-Token Prediction vs. Instruction Following
Pre-trained Large Language Models are fundamentally optimized for next-token prediction, meaning their primary training objective is to predict the next word in a sequence. This training method does not inherently equip them to follow explicit instructions. Consequently, when presented with a task-oriented prompt, such as a request for summarization, a pre-trained LLM is likely to simply continue the input text rather than executing the given command.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Human Preference Alignment via Reward Models
Limitation of Pre-trained LLMs: Next-Token Prediction vs. Instruction Following
Inference in LLMs
A development team tests two versions of a language model. They provide both models with the exact same input: 'Translate the following sentence into French: Hello, how are you?'
- Model A responds: '... I am doing well, thank you for asking. The weather is nice today.'
- Model B responds: 'Bonjour, comment allez-vous?'
Based on these outputs, what is the most likely difference in the training processes that Model A and Model B have undergone?
Classification of LLM Development Methods by Stage and Application Time
A team of AI developers is building a new large language model from scratch, aiming for it to be both knowledgeable and helpful in following user commands. Arrange the following key development stages in the typical chronological order they would be performed.
Diagnosing LLM Performance Issues
Typical Sequence of LLM Alignment Methods
Learn After
Instruction Alignment
A user interacts with a large language model that has only undergone its initial training phase on a vast corpus of text, without any subsequent fine-tuning to follow commands. The user provides the input: 'Translate the following sentence into French:'. Which of the following outputs is most characteristic of this specific type of model's behavior?
Diagnosing Language Model Output
Predicting Pre-trained Model Behavior