Flexibility of the Penalty Function

Repetition Penalty

Length Penalty

Diversity Penalty

Constraint-based Penalty

Penalty Functions Based on Hidden States

A developer is building a system to generate empathetic and cautious responses for a customer service chatbot. To achieve this, they want to implement a penalty function that discourages the model from adopting an 'overly confident' or 'assertive' internal state during the text generation process, rather than simply penalizing specific words in the final output. Which of the following penalty function designs best aligns with this goal of operating on the model's internal representations?

Comparing Penalty Function Implementations

A team is developing a text generation model and is considering two different ways to penalize undesirable outputs. Match each proposed penalty mechanism with the implementation approach it represents.