Plackett-Luce Selection Probability Formula

Optimal Policy as a Product of Reference Policy and Exponentiated Reward

Worth Function in Plackett-Luce Model

A language model's policy, which determines the probability of generating an output y given an input x, is structured to be proportional to the exponential of a reward score r(x, y). For a specific input, two potential outputs have the following reward scores:

• Output A: Reward = 3.0
• Output B: Reward = 1.0

Based on this formulation, how does the probability of generating Output A compare to the probability of generating Output B?

Analyzing Language Model Response Probabilities

A language model's policy is designed such that the probability of generating an output is proportional to the exponential of its reward score. If Output Y has a reward score that is exactly double the reward score of Output Z, it means the policy will assign exactly double the probability to Output Y compared to Output Z.

Plackett-Luce Selection Probability Formula

A system assigns a 'worth' value to potential text completions, calculated as the exponential of a reward score. Initially, three completions (A, B, C) have reward scores of 2.0, 3.0, and 4.0, respectively. If the reward score for each completion is increased by a constant value of 1.0, how does this change affect the ratio of worth between any two completions (e.g., the ratio of worth(B) to worth(A))?

Calculating Response Worth for an AI Assistant

In a system that assigns a 'worth' value to a response by taking the exponential of its reward score, doubling the reward score for a response will also double its assigned worth value.

Pros and Cons of Softmax Function

Softmax Regression (Activation)

Parameterized Softmax Layer

Plackett-Luce Selection Probability Formula

Conditional Probability Formula for Autoregressive Models using Softmax

A neural network's final layer produces the raw output scores (logits) [2.0, 1.0, 0.1] for three possible classes. To convert these scores into class probabilities, a function is applied that first exponentiates each score and then normalizes these new values by dividing each by their sum. What is the resulting probability distribution? (Values are rounded to three decimal places).

A function is used to convert a vector of raw, unnormalized scores z = [z_1, z_2, ..., z_K] into a probability distribution. This function operates by first applying the standard exponential function to each score and then normalizing these new values by dividing each by their sum. If a constant value C is added to every score in the input vector z, resulting in a new vector z' = [z_1+C, z_2+C, ..., z_K+C], how will the resulting output probability distribution be affected?

Consider two input vectors of raw scores (logits) for a 3-class classification problem: Vector A = [1, 2, 3] and Vector B = [1, 5, 10]. Both vectors are passed through a function that exponentiates each score and then normalizes the results by dividing by their sum. How will the resulting probability distribution for Vector B compare to the one for Vector A?

You’re reviewing an internal evaluation script tha...

Your team is building an internal tool that ranks ...

You’re reviewing an internal LLM evaluation pipeli...

Reconciling Training Log-Likelihood with Inference-Time Sequence Selection

Explaining a Counterintuitive Decoding Outcome Using Softmax, Next-Token Conditionals, and Sequence Log-Probability

Diagnosing a “High-Confidence Wrong Token” Bug in Autoregressive Scoring

Investigating a Production Scoring Bug: Softmax Normalization vs. Autoregressive Sequence Log-Probability

Design a Correct Sequence-Scoring Function for Autoregressive LLM Outputs

Root-Cause Analysis: Why a “More Likely” Token-by-Token Completion Loses on Total Sequence Score

Auditing a Candidate Completion Using Softmax Next-Token Probabilities and Autoregressive Log-Probability

Derivative of Softmax Cross-Entropy Loss with Respect to Logits

Numerical Overflow in Softmax Function