Learn Before
A research lab is planning to create a new instruction-tuning dataset using a team of highly skilled human experts. Their goal is to build a model capable of a novel, complex reasoning task. Based on the inherent limitations of manual data generation, which of the following proposed tasks would be the most difficult for the expert annotators to execute consistently and reliably at scale?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Example of Human Annotation Challenge: Long Document Analysis
Critique of an Expert Annotation Plan
A research lab is planning to create a new instruction-tuning dataset using a team of highly skilled human experts. Their goal is to build a model capable of a novel, complex reasoning task. Based on the inherent limitations of manual data generation, which of the following proposed tasks would be the most difficult for the expert annotators to execute consistently and reliably at scale?
Annotation Feasibility for a Legal AI