Learn Before
Instruction Sampling for Diversity in Self-Instruct
In each iteration of the Self-Instruct process, a small subset of instructions is selected from the task pool to be used as prompts for generating new instructions. To maintain diversity in the generated tasks, this selection can include a mix of both the initial, human-written seed instructions and the instructions previously generated by the Large Language Model.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Instruction Sampling for Diversity in Self-Instruct
Example of a Prompt Template for Instruction Generation in Self-Instruct
Sample Generation in Self-Instruct
An AI development team provides a large language model with a prompt containing several existing task instructions, such as 'Translate this sentence into French' and 'Write a poem about the ocean.' The prompt then asks the model to generate a new, distinct instruction based on the examples provided. What is the primary function of including the existing instructions in the prompt?
Automated Task Creation for a Marketing Dataset
Analyzing a Flawed Prompt for Instruction Generation
Learn After
Diagnosing Dataset Generation Issues
A research team is using a self-instruction method to generate a large dataset of tasks. In their process, for each new generation step, they exclusively sample from the small, initial set of human-written examples to prompt the language model. What is the most probable outcome for the final dataset if they follow this strategy?
Rationale for Mixed Instruction Sampling