Human Preference Alignment via Reward Models

Limitation of Pre-trained LLMs: Next-Token Prediction vs. Instruction Following

Inference in LLMs

A development team tests two versions of a language model. They provide both models with the exact same input: 'Translate the following sentence into French: Hello, how are you?'

• Model A responds: '... I am doing well, thank you for asking. The weather is nice today.'
• Model B responds: 'Bonjour, comment allez-vous?'

Based on these outputs, what is the most likely difference in the training processes that Model A and Model B have undergone?

Classification of LLM Development Methods by Stage and Application Time

A team of AI developers is building a new large language model from scratch, aiming for it to be both knowledgeable and helpful in following user commands. Arrange the following key development stages in the typical chronological order they would be performed.

Diagnosing LLM Performance Issues

Typical Sequence of LLM Alignment Methods