To generate a single vector representing an entire prompt-response sequence in a reward model, the sequence is processed from left to right using forced decoding. Because language modeling restricts each position to only accessing its left context, the output from the top-most Transformer layer at the first position cannot encapsulate the full sequence. To resolve this, a special symbol, such as $\langle \backslash s \rangle$, is appended to the end of the sequence. The corresponding output vector from the Transformer layer stack at this final position is then used as the comprehensive representation of the entire sequence.