Learn Before
Application Formula for Fine-Tuned BERT Models
Once the fine-tuning optimization is complete, the resulting optimized parameters, and , are utilized to make predictions on new data. For a specific task that the model was tuned for, applying the fine-tuned model during inference is mathematically represented by the formula . This expression indicates that an input is first processed by the base BERT model using its tuned parameters , and the resulting representations are then passed into the task-specific prediction network, which utilizes its own tuned parameters .
0
1
Tags
Foundations of Large Language Models
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Fine-Tuning as Maximum Likelihood Estimation
A machine learning engineer is adapting a large, pre-trained language model for a new text classification task. They have a labeled dataset D containing pairs of text inputs (x) and their correct labels (y_gold). The engineer formulates the following objective for the adaptation process, where θ represents the model parameters which are initialized randomly:
What is the primary conceptual error in this formulation for the specific goal of adapting the pre-trained model?
Notation for Parameters in the Fine-Tuning Process
Comparing Optimization Objectives in Model Training
The objective for fine-tuning a pre-trained model is formally expressed as: Match each component of this objective function to its correct description.
Application Formula for Fine-Tuned BERT Models