Hypothesis Validation from Training to Training Dev Subset Performance
Question: When both the training and training dev sets contain car audio, what specific finding about the performance on these car audio subsets would validate the hypothesis that collecting more car audio data will help?
Sample answer: The hypothesis is validated if the system performs well on the car audio in the training set but performs poorly on the car audio in the training dev set.
Key points:
- System does well on the car data subset in the training set.
- System does poorly on the car data subset in the training dev set.
Rubric: The response should state that the model performs well/does well on the car data in the training set, but does not perform well/fails on the car data in the training dev set.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Related
What does it indicate when a model performs well on car audio in the training set but poorly on car audio in the training dev set?
If both your training set and training dev set contain car audio, you should evaluate your system's performance specifically on that car-audio subset.
If a model does well on car audio in the training set but poorly on car audio in the training dev set, this validates the hypothesis that getting more _____ data would help.
When training and training dev sets both include car-recorded audio, what action should you take to investigate the data mismatch hypothesis?
If a model performs well on car audio in the training set but poorly on car audio in the training dev set, this further validates the hypothesis that getting more car data would help.
If the system does well on car data in the training set but not on car data in the _____, this further validates the mismatch hypothesis.
Match each observation about car-audio subset performance to its implication for the data mismatch hypothesis.
Order the diagnostic steps for using a shared subset (e.g., car audio) to check the data mismatch hypothesis.
What conclusion should you draw if your model achieves high accuracy on car audio in the training set but low accuracy on car audio in the training dev set?
Checking performance on a shared subset in the training and training dev sets can only refute—never further validate—a data mismatch hypothesis.
Ng recommends double-checking the system's performance on the car-audio _____ when both the training and training dev sets include car-recorded audio.
Match each key term in the mismatch hypothesis checking procedure to its correct description.
Order the reasoning steps for deciding whether to collect more car data, starting from suspecting a mismatch to reaching a validated conclusion.
Analyzing Mismatch Hypotheses via Training and Training Dev Subsets
Diagnosing Speech Recognition Performance in Car Audio Subsets
Hypothesis Validation from Training to Training Dev Subset Performance