Learn Before
Evaluating Segmentation Strategy Suitability
A development team is building a reward model for two distinct tasks: summarizing lengthy legal contracts and generating Python code snippets. They decide to use a uniform fixed-length segmentation strategy, breaking down the generated text for both tasks into 128-token chunks for human rating. Evaluate the appropriateness of this uniform segmentation strategy for these two different tasks. Justify your reasoning by considering the potential impact on the quality of the human feedback for each task.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A team is developing a reward model to assess the quality of multi-paragraph essays. To do this, they segment each essay into non-overlapping chunks of exactly 150 words for human evaluation. Which of the following describes the most significant weakness inherent in this specific segmentation strategy?
Evaluating Segmentation Strategy Suitability
Analyzing a Segmentation Strategy for Code Evaluation