In many inference problems, one can easily improve the accuracy of the predicted $\hat{Y}$ by fitting a more complicated $\hat{f}$, but the question is whether to do that. If having a more complicated $\hat{f}$ makes it difficult to interpret the results, it is better to use the simpler model. This is one of the main reasons that most inferential studies assume the linearity of the model. Conversely, if the objective is not inference but getting the most accurate prediction, we can use more sophisticated functional forms for $\hat{f}$.

University of Michigan - Ann Arbor

In both prediction and inference, we estimate a function $f$ that maps the values of predictors $X$ to outcome $Y$.
In prediction, our goal is to estimate the function $f$ based on the available values of $X$ and corresponding $Y$ so that we can use $f$ to predict the values of $Y$ for the values of $X$ for which we don't know the corresponding values of $Y$. So, the function $\hat{f}$ can be treated as a black box, and our main objectives are to improve the accuracy and precision of the prediction.
In inference, we already know all the corresponding values of $X$ and $Y$, and we're interested in understanding the characteristics of the function $f$ that maps the values of $X$ to their corresponding values of $Y$. So, $\hat{f}$ cannot be treated as a black box and our main objective is to find the exact functional form of $\hat{f}$.

Inference vs. prediction in statistical learning

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Trade-off between interpretability and accuracy of prediction in statistical learning

In deep learning, the phase where a trained model, such as $$\hat{\mathbf{w}}^	op \mathbf{x} + \hat{b}$$, is used to estimate outputs for new, unseen examples is often referred to as inference. However, this usage is a misnomer that can cause confusion with the statistics literature, where inference more broadly denotes any conclusion drawn from evidence, particularly parameter inference. To avoid this ambiguity, it is more precise to use the term prediction for the process of estimating labels for new instances.

Learn Before

Related