Learn Before
Typical Sequence of LLM Alignment Methods
After a Large Language Model completes its initial pre-training stage, alignment is typically achieved by applying three methods in a specific sequence. First, Supervised Fine-Tuning (SFT) is performed to adapt the model to specific instructions. Second, Reinforcement Learning from Human Feedback (RLHF) is utilized to align the model with complex human preferences and values. Finally, during the inference stage, prompting techniques are employed to dynamically guide the model's behavior for specific tasks.
0
1
Tags
Foundations of Large Language Models
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Human Preference Alignment via Reward Models
Limitation of Pre-trained LLMs: Next-Token Prediction vs. Instruction Following
Inference in LLMs
A development team tests two versions of a language model. They provide both models with the exact same input: 'Translate the following sentence into French: Hello, how are you?'
- Model A responds: '... I am doing well, thank you for asking. The weather is nice today.'
- Model B responds: 'Bonjour, comment allez-vous?'
Based on these outputs, what is the most likely difference in the training processes that Model A and Model B have undergone?
Classification of LLM Development Methods by Stage and Application Time
A team of AI developers is building a new large language model from scratch, aiming for it to be both knowledgeable and helpful in following user commands. Arrange the following key development stages in the typical chronological order they would be performed.
Diagnosing LLM Performance Issues
Typical Sequence of LLM Alignment Methods