Diagnosing a Model Training Issue
Assuming the data being used is appropriate and the model's architecture is sound, analyze the situation described in the case study. Which part of the fundamental training procedure is most likely failing, and why does this failure lead to the observed outcome?
0
1
Tags
Data Science
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
General Objective for Parameter Optimization via Loss Minimization
BERT Training Process
Diagnosing a Model Training Issue
A neural network is trained by repeatedly showing it examples from a dataset. Arrange the following core steps of a single training iteration into the correct logical sequence.
During the training of a neural network, an optimization algorithm iteratively adjusts the model's parameters. If the value of the loss function is consistently decreasing over many iterations, what is the most direct interpretation of this trend?
Standard Optimization Objective for Transformer Language Models