Having three evaluation metrics makes it harder for you to quickly choose between two different algorithms, and will slow down the speed with which your team can iterate. True/False?

Typically, the $$R^2$$ score is sufficient for evaluating a regression model. Alternative evaluation metrics include:
- Mean Absolute Error (MAE): The absolute difference between target and predicted values, representing the expected value of $$L_1$$ norm loss.
- Mean Squared Error (MSE): The squared difference between target and predicted values, representing the expected value of $$L_2$$ norm loss.
- Median Absolute Error: Uses the median of the error distribution rather than the mean, making it more robust to outliers.

Characteristics of Regression Evaluation Metrics

When dealing with financial datasets such as stock or house prices, practitioners generally care more about the relative error $$\frac{y - \hat{y}}{y}$$ than the absolute error $$y - \hat{y}$$. For instance, a prediction error of \$100,000 is massive when estimating the price of a typical \$125,000 rural house, but the exact same absolute error represents a stunningly accurate prediction for a median house price exceeding \$4 million in a wealthy neighborhood.

Relative Error in Price Prediction

Regression model evaluation metrics are quantitative measures used to assess how well a model's predictions match the actual observed data. Common metrics include Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), the Coefficient of Determination ($$R^2$$), and Adjusted $$R^2$$.

University of Michigan - Ann Arbor

Johns Hopkins University

Google

A regression model predicts a value $$Y$$ given a set of $$X$$ variables. In the regression model $$Y \sim X$$, we measure the effect of $$X$$ on $$Y$$, i.e., we measure the association between changes in values of $$X$$ and changes in values of $$Y$$. Analyzing a regression model is a method used to solve a regression problem.

Reference: James, G., Witten, D., Hastie, T., & Tibshirani, R. (2017). An introduction to statistical learning: with applications in R. New York: Springer.

Regression Model

In the regression model $Y \sim X$, to control for the effect of confounder $Z$ (where $X \leftarrow Z \rightarrow Y$) we can use the regression model $Y \sim X + Z$, i.e., taking the average effect of $X$ on $Y$ across differnt values of $Z$.

Learn Before

Related

Learn After