Learn Before
Speech Recognition Scoring Error
Case context: You are building a speech recognition system. For a given audio clip, the correct transcription S* is "I love machine learning". However, your system outputs Sout = "I love matching learning". You run an Optimization Verification test and find that ScoreA(S*) is 0.82 and ScoreA(Sout) is 0.85.
Question: Based on the Optimization Verification test results, what is the source of the inference failure, and what should you prioritize to fix it?
Sample answer: The source of the failure is an objective (scoring) function problem, because the score for the correct output (0.82) is less than the score for the incorrect output (0.85). I should prioritize improving the learning algorithm that estimates the score, ScoreA(S), rather than the search algorithm.
Key points:
- Diagnose as an objective (scoring) function problem.
- Identify that ScoreA(S*) <= ScoreA(Sout).
- Prioritize improving the learning algorithm.
Rubric: The learner must correctly diagnose the objective function problem and prescribe fixing the learning algorithm.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Data Science
Foundations of Large Language Models Course
Computing Sciences
D2L
Dive into Deep Learning @ D2L
Machine Learning
Deep Learning
Supervised Learning
Machine Learning Yearning @ DeepLearning.AI
Related
Cross-entropy loss
Logistic Regression Cost Function
A machine learning model is being trained for a prediction task. A key metric, the objective function, is tracked over time. The value of this function represents the magnitude of the model's error. A graph of this process shows the objective function's value consistently decreasing as the number of training iterations increases. What is the most accurate interpretation of this trend?
Diagnosing Model Training Issues
Calculating and Interpreting a Model's Objective Function
Surrogate Objective
Loss Function
Differentiable Objectives
Second-Order Optimization Algorithm
Objective Function Curvature
Convex Quadratic Objective Function
Identifying an Objective Function Problem
Improving the Search Algorithm
An objective or scoring function can be the source of an inference failure when it does not assign a _____ score to the correct output than to the system output.
Optimization Verification Test Scenarios
Diagnosing an Objective Function Failure
Responding to Objective Function Failures
Speech Recognition Scoring Error
Fixing Scoring Function Inaccuracies
Purpose of the Objective Function
Optimization Verification Test Result