To illustrate the process of ranking Large Language Model generations, suppose a model produces four different responses regarding environmental sustainability, denoted as $\mathbf{y}_1$ through $\mathbf{y}_4$. For instance, $\mathbf{y}_1$ suggests using electric vehicles, $\mathbf{y}_2$ advocates minimalism, $\mathbf{y}_3$ proposes going off-grid, and $\mathbf{y}_4$ recommends supporting local farms. Rather than assigning an independent numerical score to each option, human annotators compare the set and arrange them by preference. A possible ranking outcome might be $\mathbf{y}_1 \succ \mathbf{y}_4 \succ \mathbf{y}_2 \succ \mathbf{y}_3$, demonstrating that $\mathbf{y}_1$ is the most favored response while $\mathbf{y}_3$ is the least favored.