Optimization Verification does not require a truly optimal output. If a superior output is available, such as a human pilot trajectory that is better than the current learning algorithm output, the test can still indicate whether improving the optimization algorithm or the scoring function is more promising.

Google

The Optimization Verification test compares the score of a known correct output with the score of the system output. If the correct output scores higher, blame the optimization or search algorithm; otherwise, blame the scoring function computation.

Optimization Verification Test

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

In practice, apply the Optimization Verification test across errors in the dev set. Each error where the correct output scores higher is marked as an optimization-algorithm error; each error where it does not is counted as a scoring-function error.

Applying Optimization Verification Across Dev Set Errors

A common AI design pattern is to first learn an approximate scoring function and then use an approximate maximization algorithm. Recognizing this pattern lets one use Optimization Verification to understand the source of errors.

Approximate Scoring Function Plus Approximate Maximization Pattern

For machine translation, compute a score for possible translations and use heuristic search because the set of possible English sentences is too large. If the correct translation scores above the system translation, attribute the error to approximate search; otherwise, attribute it to the score computation.

Optimization Verification for Machine Translation

For a reinforcement learning system whose trajectory is worse than a human pilot trajectory, compare the reward assigned to the human trajectory with the reward assigned to the algorithm trajectory. If the human trajectory scores higher, improving the reinforcement learning algorithm is worthwhile; if it does not, improve the reward function.

Learn Before

Related