Learn Before
Classification of LLM Development Methods by Stage and Application Time
LLM development methods are classified based on the stage they belong to—pre-training or alignment—and when they are applied. The pre-training stage involves the initial model training. The subsequent alignment stage includes methods that modify the model's parameters during training and fine-tuning, such as Instruction Alignment (e.g., SFT) and Human Preference Alignment (e.g., RLHF). Another category of alignment methods, like prompting, is applied during inference to guide model behavior without altering its weights.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Human Preference Alignment via Reward Models
Limitation of Pre-trained LLMs: Next-Token Prediction vs. Instruction Following
Inference in LLMs
A development team tests two versions of a language model. They provide both models with the exact same input: 'Translate the following sentence into French: Hello, how are you?'
- Model A responds: '... I am doing well, thank you for asking. The weather is nice today.'
- Model B responds: 'Bonjour, comment allez-vous?'
Based on these outputs, what is the most likely difference in the training processes that Model A and Model B have undergone?
Classification of LLM Development Methods by Stage and Application Time
A team of AI developers is building a new large language model from scratch, aiming for it to be both knowledgeable and helpful in following user commands. Arrange the following key development stages in the typical chronological order they would be performed.
Diagnosing LLM Performance Issues
Typical Sequence of LLM Alignment Methods
Learn After
Classifying an Alignment Approach
Match each language model development technique with the description that best classifies its stage, application time, and effect on the model's parameters.
A development team is working with a large, pre-existing language model. They need to make the model generate outputs in a specific structured format for a new, specialized task. However, they are operating under a strict constraint: they cannot alter the model's saved parameters due to computational limitations. Which of the following strategies should they employ?