Google

LLM development methods are classified based on the stage they belong to—pre-training or alignment—and when they are applied. The pre-training stage involves the initial model training. The subsequent alignment stage includes methods that modify the model's parameters during training and fine-tuning, such as Instruction Alignment (e.g., SFT) and Human Preference Alignment (e.g., RLHF). Another category of alignment methods, like prompting, is applied during inference to guide model behavior without altering its weights.

Classification of LLM Development Methods by Stage and Application Time

A development team has a large, general-purpose language model. To make it safer for public use, they implement a system where every user prompt is automatically prepended with a detailed set of instructions, such as 'You are a helpful and harmless assistant. Do not generate unsafe content.' This is done for every new conversation without making any permanent changes to the model's underlying structure. Based on this approach, which category of alignment method is the team using? Justify your answer.

Classifying an Alignment Approach

Match each language model development technique with the description that best classifies its stage, application time, and effect on the model's parameters.

A development team is working with a large, pre-existing language model. They need to make the model generate outputs in a specific structured format for a new, specialized task. However, they are operating under a strict constraint: they cannot alter the model's saved parameters due to computational limitations. Which of the following strategies should they employ?

Learn Before

Related