Reward-Weighted Probability Distribution

Consider a scenario where for a given input (\mathbf{x}), there are only two possible outputs, (\mathbf{y}_1) and (\mathbf{y}2). A reference model assigns probabilities (\pi{\text{ref}}(\mathbf{y}1|\mathbf{x}) = 0.6) and (\pi{\text{ref}}(\mathbf{y}_2|\mathbf{x}) = 0.4). A reward function gives scores (r(\mathbf{x}, \mathbf{y}1) = 2) and (r(\mathbf{x}, \mathbf{y}2) = 1). Assuming the scaling factor (\beta) is 1, what is the value of the normalization factor (Z(\mathbf{x})), which is calculated as (Z(\mathbf{x}) = \sum{\mathbf{y}} \pi{\text{ref}}(\mathbf{y}|\mathbf{x}) \exp(r(\mathbf{x}, \mathbf{y})))?

Consider the calculation of a normalization factor using the formula: $Z(\mathbf{x}) = \sum_{\mathbf{y}} \pi_{\theta_{\text{ref}}}(\mathbf{y}|\mathbf{x}) \exp \left(\frac{1}{\beta}r(\mathbf{x}, \mathbf{y})\right)$ If the reward function (r(\mathbf{x}, \mathbf{y})) consistently returns a value of 0 for all possible outputs (\mathbf{y}), the normalization factor (Z(\mathbf{x})) will always be equal to 1.

Impact of Scaling Factor on Normalization