Learn Before
Notation for Ground Truth Labels in Segment Classification
In the context of segment-based classification tasks, the ground truth label for a segment is denoted by . This label typically belongs to a set of discrete values, such as , representing binary categories like 'aligned' or 'misaligned'.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Training Reward Models with Classification Loss for Segment Alignment
A team is developing a safety filter for a language model. Their goal is to prevent the model from generating text that falls into several strictly prohibited categories (e.g., revealing private data, generating hate speech). For fine-grained feedback, they evaluate each model response by breaking it into smaller segments. Which evaluation strategy would be most effective for this specific goal, and why?
Segment Evaluation Methods
Improving Content Moderation Feedback
Notation for Ground Truth Labels in Segment Classification
Learn After
Consider a task where a language model's response is divided into segments, and each segment is classified by a human annotator. The set of possible labels is {1, -1}, where 1 represents an 'aligned' segment and -1 represents a 'misaligned' segment. An annotator reviews the second segment of a response, which contains factually incorrect information, and determines it is 'misaligned'. How would the ground truth label for this second segment be correctly represented?
In a task where text segments are classified as 'aligned' (represented by 1) or 'misaligned' (represented by -1), the notation for a specific segment represents the model's predicted probability that the segment is 'aligned'.
In a scenario where a language model's response is broken into segments for evaluation, a human reviewer assigns a definitive label to each segment from a predefined set (e.g., {1 for 'helpful', -1 for 'unhelpful'}). This definitive, correct label for a single segment is known as the ground truth and is represented by the notation ____.