A researcher is fine-tuning a pre-trained language model on a new dataset. They represent the optimization objective using the following simplified notation:

$\tilde{\theta} = \arg \max_{\theta} \sum_{(\mathbf{x},\mathbf{y})\in\mathcal{D}} \log \mathrm{Pr}_{\theta}(\mathbf{y}|\mathbf{x})$

Based on standard conventions in this field, what is the most accurate interpretation of the parameters θ being optimized in this formula?

When the supervised fine-tuning objective is written as $\tilde{\theta} = \arg \max_{\theta} \sum_{(\mathbf{x},\mathbf{y})\in\mathcal{D}} \log \mathrm{Pr}_{\theta}(\mathbf{y}|\mathbf{x})$, the parameters denoted by $\theta$ are typically initialized from a random distribution before the optimization process begins.

Potential Misinterpretation of Fine-Tuning Notation