Learn Before
Approximate Scoring Followed by Approximate Search
Many machine learning systems work in two stages: they estimate how good each candidate option is, and then they use a search procedure that tries to pick a strong option without guaranteeing the exact best one. Spotting this setup helps diagnose whether mistakes mainly come from the scoring step or from the search step.
0
1
Tags
Data Science
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Classifying development-set mistakes with optimization verification
Approximate Scoring Followed by Approximate Search
Checking Whether a Translation Error Comes from Search or Scoring
Checking Reward Quality in Reinforcement Learning
A useful human output for optimization verification
What does this optimization check suggest when the reference candidate scores higher than the algorithm’s output?
Using the Optimization Verification Test
Optimization Check Symbols
How to carry out an optimization verification check
Using the Optimization Verification Check
Debugging a Translation Decoder
When to Attribute an Error to Optimization
When Is an Optimization Check Useful?
Checking the Source of a Performance Problem
Optimization checks can isolate the source of a training problem
Learn After
Which two approximations are combined in the standard score-and-search design pattern?
True or False: If a scoring rule does not depend on a particular input x, the notation Score_x(.) can be written simply as Score(.).
When a system assigns scores to many candidate outputs and then searches for the highest-scoring one, what diagnostic check helps evaluate the search step?
Which pair describes a common AI workflow for choosing outputs?
If a system first estimates a score approximately and then chooses the best output approximately, the Optimization Verification test can help identify whether the main error comes from scoring or from the maximization step.
If a scoring expression has no input-dependent term, it collapses to just _____.
Match each item in the route-ranking example to the role it plays in the pattern.
Order the steps in a score-then-search machine learning workflow.
In a two-stage chess engine, which part acts as the approximate maximization algorithm?
In the usual AI pipeline, the search step always returns the highest-scoring output exactly.
Many machine learning systems optimize an approximate _____ with an approximate search routine.
Match each part of the approximate scoring plus approximate maximization pattern to its description.
Order the steps for deciding whether to use the search-and-score diagnostic on a model.
Two-Stage Search for Better Predictions
Tracing a Poor Route in a Learned Scoring System
Why Identifying the Approximate Score-and-Search Pattern Helps