A predictive model was trained to estimate a student's final exam score based on their hours spent studying. Analyze the table below, which shows the actual scores and the model's predicted scores for four different students. Based on this data, describe the model's performance, paying close attention to any patterns in its errors.

Google

In predictive modeling, ŷ (pronounced 'y-hat') represents the predicted or estimated value for a dependent variable, y. It is the output generated by a model based on a given set of input features. The primary objective during model training is to adjust the model's parameters so that its predictions, ŷ, are as close as possible to the actual, observed values of y.

Predicted Value (ŷ)

In machine learning, an objective function quantifies the goal of an algorithm, which is typically to minimize an error measure (a cost function) or maximize a probability function. For supervised learning problems like classification and prediction, an error function is defined to capture the discrepancy between the known correct outputs and the model's predicted outputs. The learning process then aims to find model parameters that minimize this function.

Objective Function in Machine Learning

Simple linear regression commonly uses a least squares approach, where the linear model attempts to minimize the residual distance between the actual points and the predicted line. Consequently, its residual is defined as:
$$e_i = y_i -\hat{y}_i$$

where $y_i$ represents the actual ith response value and $\hat{y}_i$ represents the ith response value of the linear model. 

Least Squares Approach

The predicted value, denoted as $\hat{y}$, is formally determined as the argument that optimizes a specific objective function. This is expressed in the general form `$\hat{y}$ = arg ...`, where `arg` typically represents either `argmin` (argument of the minimum) or `argmax` (argument of the maximum). The choice between `argmin` and `argmax` depends on whether the objective is to minimize a loss function or maximize a likelihood function, respectively.

Formal Definition of the Predicted Value (ŷ)

A real estate company uses a machine learning model to estimate the market value of houses. For a specific house with 3 bedrooms and 2,000 square feet of living space, the model calculates an estimated value of $450,000. The house later sells for an actual price of $465,000. In the context of this predictive model, what does the $450,000 figure represent?

A data scientist builds a model to predict a student's final exam score based on their hours spent studying. For one student who studied for 10 hours, the model predicts a score of 85. The student's actual score on the exam was 88. Identify the predicted value and the actual value in this scenario, and explain what the difference between these two values indicates about the model's performance for this specific prediction.

Analyzing Model Predictions

The expression $\text{Pr}^t(y|c, z)$ denotes the conditional probability of an output variable $y$ given two conditioning variables, $c$ and $z$. The superscript $t$ can indicate a specific state or version of the model, such as at a particular training step. This formulation is central to probabilistic predictive models, where it is used to quantify the likelihood of a particular outcome based on given context and other factors.

Conditional Probability Pr^t(y|c, z)

Analyzing a Predictive Model's Performance

Linear regression uniquely presents an optimization problem with a direct analytic solution. By subsuming the bias term into the weight vector $$\mathbf{w}$$—which is done by appending a column of all $$1$$s to the design matrix $$\mathbf{X}$$—the training objective becomes minimizing $$\|\mathbf{y} - \mathbf{X}\mathbf{w}\|^2$$. Setting the derivative of this loss with respect to $$\mathbf{w}$$ equal to $$0$$ yields the intermediate derivative formula $$\partial_{\mathbf{w}} \|\mathbf{y} - \mathbf{X}\mathbf{w}\|^2 = 2 \mathbf{X}^	op (\mathbf{X} \mathbf{w} - \mathbf{y}) = 0$$, which directly gives the normal equation $$\mathbf{X}^	op \mathbf{y} = \mathbf{X}^	op \mathbf{X} \mathbf{w}$$. Solving for $$\mathbf{w}$$ provides the optimal parameters: $$\mathbf{w}^* = (\mathbf{X}^	op \mathbf{X})^{-1}\mathbf{X}^	op \mathbf{y}$$. This closed-form solution exists only if the design matrix has full rank, ensuring that the matrix $$\mathbf{X}^	op \mathbf{X}$$ is invertible.

Learn Before

Related