Question:
Explain why an objective (scoring) function failure occurs and what steps should be taken to fix it, referencing the Optimization Verification test.

Sample answer:
An objective function failure occurs when the scoring function, such as ScoreA(S), assigns a lower or equal score to the correct output (S*) compared to the incorrect system output (Sout). This means the function fails to recognize the correct answer, causing an inference failure. To fix this, developers should focus on improving the learning algorithm that estimates or approximates the score, rather than trying to improve the search algorithm.

Key points:
- Failure occurs when ScoreA(S*) <= ScoreA(Sout).
- Indicates the scoring function does not give a strictly higher score to the correct output.
- Focus on improving the learning algorithm, not the search algorithm.

Rubric:
Evaluates understanding of when an objective function fails and the appropriate corrective action.

Google

An objective or scoring function can be the source of an inference failure when it does not assign a higher score to the correct output than to the system output. In that case, the learning algorithm that estimates the score should be improved rather than the search algorithm.

Objective Function

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

The cross-entropy loss function works very well for models that predict binary classes (aka the output is between 0 and 1). It is defined as -[y*log(y-hat) +(1-y)*log(1-(y-hat))]. If y=0 the left side of the function is dropped and the right side, -log(1-(y-hat)), is used. Otherwise if y=1 the right side of the function is dropped and it uses -log(y-hat). In both instances this loss function encourages probabilities that are close to the true probability. 

Cross-entropy loss

To train the parameters W and B of the logistic regression model, you need to define a cost function.

$$J(w, b) = \frac{1}{m} \sum_{i=1}^{m} L(\hat{y}^{(i)}, y^{(i)})$$

$$=-\frac{1}{m} \sum_{i=1}^{m} [y^{(i)}log(\hat{y}^{(i)}) + (1 - y^{(i)})log(1 - \hat{y}^{(i)})]$$

This loss function is Convex.

Logistic Regression Cost Function

A machine learning model is being trained for a prediction task. A key metric, the objective function, is tracked over time. The value of this function represents the magnitude of the model's error. A graph of this process shows the objective function's value consistently decreasing as the number of training iterations increases. What is the most accurate interpretation of this trend?

An engineer is training a model to predict housing prices. After running the training process for several hours, they plot the value of the model's error measurement over time. They observe that the error value remains very high and does not decrease, staying almost flat throughout the entire process. Based on this observation, analyze the effectiveness of the training process and explain what this trend indicates about the model's ability to achieve its primary goal.

Diagnosing Model Training Issues

A machine learning model is designed to predict the price of a product. For a small sample of three products, the model predicts prices of [$55, $90, $125], while the actual prices are [$50, $100, $120]. The model's performance is measured by an objective function defined as the average of the squared differences between the predicted and actual values. First, calculate the value of this objective function for the given sample. Second, explain what a lower value of this function would signify about the model's future predictions.

Calculating and Interpreting a Model's Objective Function

When a primary objective function—such as the error rate in classification—is difficult to optimize directly due to non-differentiability or other mathematical complications, machine learning models instead optimize a surrogate objective. This proxy function is chosen because it is easier to compute gradients for while still aligning with the ultimate goal.

Surrogate Objective

A loss function is a specific type of objective function where the convention is that lower values indicate better model performance. Optimization algorithms actively seek to minimize the loss function to improve the model; any objective where higher is better can be converted into a loss function by simply flipping its sign.

Loss Function

A fundamental requirement for training modern machine learning and deep learning models is the use of differentiable objectives. Because the optimization process typically relies on gradient-based methods, such as minibatch stochastic gradient descent, the objective function (or loss function) must be mathematically differentiable with respect to the model's parameters. This differentiability allows the optimization algorithm to compute gradients, which provide the direction and magnitude of the parameter updates needed to minimize the error and improve the model's predictive performance.

Differentiable Objectives

First-order optimization algorithms rely solely on the value and gradient of the objective function. In contrast, second-order optimization algorithms also utilize information about the function's curvature, often represented by the Hessian matrix. By accounting for curvature, these methods can automatically adjust the optimization step, providing a way to circumvent the difficulties of manually tuning a learning rate.

Second-Order Optimization Algorithm

In the context of optimization, the curvature of an objective function refers to the rate at which its gradient changes, conceptually captured by its second-order derivatives. Geometrically, curvature indicates how rapidly the surface of the objective function bends. Understanding this property provides useful intuition for adjusting optimization step sizes: in regions of high curvature where the gradient changes quickly, smaller step sizes help avoid overshooting the optimal solution or diverging; in regions of low curvature, larger step sizes can safely accelerate progress. While computing curvature directly is often too computationally expensive for deep learning, it forms the theoretical foundation for designing advanced adaptive optimization algorithms that automatically adjust their learning rates.

Objective Function Curvature

A convex quadratic objective function is defined by the general mathematical form $$h(\mathbf{x}) = \frac{1}{2} \mathbf{x}^	op \mathbf{Q} \mathbf{x} + \mathbf{x}^	op \mathbf{c} + b$$, where the matrix $$\mathbf{Q}$$ is positive definite ($$\mathbf{Q} \succ 0$$). Because $$\mathbf{Q}$$ possesses strictly positive eigenvalues, this function has a unique global minimizer located at $$\mathbf{x}^* = -\mathbf{Q}^{-1} \mathbf{c}$$. The function can be rewritten by centering it around this minimizer, yielding $$h(\mathbf{x}) = \frac{1}{2} (\mathbf{x} - \mathbf{Q}^{-1} \mathbf{c})^	op \mathbf{Q} (\mathbf{x} - \mathbf{Q}^{-1} \mathbf{c}) + b - \frac{1}{2} \mathbf{c}^	op \mathbf{Q}^{-1} \mathbf{c}$$. Furthermore, its gradient is given by $$\partial_{\mathbf{x}} h(\mathbf{x}) = \mathbf{Q} (\mathbf{x} - \mathbf{Q}^{-1} \mathbf{c})$$, which geometrically represents the distance from the point $$\mathbf{x}$$ to the minimizer scaled by the curvature matrix $$\mathbf{Q}$$.

Convex Quadratic Objective Function

Identifying an Objective Function Problem

Improving the Search Algorithm

An objective or scoring function can be the source of an inference failure when it does not assign a _____ score to the correct output than to the system output.

Optimization Verification Test Scenarios

Diagnosing an Objective Function Failure

Responding to Objective Function Failures

Case context:
You are building a speech recognition system. For a given audio clip, the correct transcription S* is "I love machine learning". However, your system outputs Sout = "I love matching learning". You run an Optimization Verification test and find that ScoreA(S*) is 0.82 and ScoreA(Sout) is 0.85.

Question:
Based on the Optimization Verification test results, what is the source of the inference failure, and what should you prioritize to fix it?

Sample answer:
The source of the failure is an objective (scoring) function problem, because the score for the correct output (0.82) is less than the score for the incorrect output (0.85). I should prioritize improving the learning algorithm that estimates the score, ScoreA(S), rather than the search algorithm.

Key points:
- Diagnose as an objective (scoring) function problem.
- Identify that ScoreA(S*) <= ScoreA(Sout).
- Prioritize improving the learning algorithm.

Rubric:
The learner must correctly diagnose the objective function problem and prescribe fixing the learning algorithm.

Speech Recognition Scoring Error

Question:
If the Optimization Verification test reveals that ScoreA(S*) ≤ ScoreA(Sout), what specific component of your machine learning system should you work on improving?

Sample answer:
You should work on improving the learning algorithm that learns or approximates the scoring function.

Key points:
- The learning algorithm
- Estimation or approximation of the scoring function

Rubric:
Correctly identifies the learning algorithm or the estimation of the scoring function.

Learn Before

Related