Google

Instruction fine-tuning is considered a sample-efficient method because it effectively adapts a model's behavior using a significantly smaller set of training examples than what is required for pre-training.

Instruction Fine-Tuning as a Sample-Efficient Method

Based on the scenario provided, which training strategy should the startup choose to make the most effective use of their available resources? Justify your recommendation by explaining the relationship between the size of their dataset and the proposed training approach.

AI Startup Strategy for Legal Document Summarization

A language model is first trained on a massive dataset containing trillions of words from the public internet. Subsequently, it is adapted to follow user commands and answer questions helpfully using a much smaller, carefully curated dataset of only a few thousand examples. Analyze why the second training phase can be successful with a dramatically smaller dataset compared to the first.

Data Requirements in Model Training Phases

A research lab has access to a large, general-purpose language model that was developed by training it on a vast and diverse collection of text from the internet. The lab's goal is to make this model specialized for generating safe and helpful medical advice. They have a limited budget, allowing them to create only a few thousand high-quality examples of medical questions paired with ideal, safe answers. Which of the following is the most effective and resource-conscious strategy for the lab to 

Learn Before

Related