Representation-based Repetition Penalty

A developer wants to ensure a language model generates multi-paragraph text that maintains a consistent theme, penalizing outputs that start on one topic and then drift into an unrelated one. Why is a penalty function that assesses the model's internal hidden states generally more effective for this specific task than a function that only evaluates the final, complete text?

Designing a Penalty Function for Safe AI

A researcher aims to guide a language model to generate text with a consistently positive sentiment, penalizing it the moment its internal thought process begins to drift towards negativity, even before negative words are explicitly written. Which approach to designing a penalty function is most suitable for this real-time, internal-state intervention?