In both prediction and inference, we estimate a function $f$ that maps the values of predictors $X$ to outcome $Y$.
In prediction, our goal is to estimate the function $f$ based on the available values of $X$ and corresponding $Y$ so that we can use $f$ to predict the values of $Y$ for the values of $X$ for which we don't know the corresponding values of $Y$. So, the function $\hat{f}$ can be treated as a black box, and our main objectives are to improve the accuracy and precision of the prediction.
In inference, we already know all the corresponding values of $X$ and $Y$, and we're interested in understanding the characteristics of the function $f$ that maps the values of $X$ to their corresponding values of $Y$. So, $\hat{f}$ cannot be treated as a black box and our main objective is to find the exact functional form of $\hat{f}$.

Inference vs. prediction in statistical learning

The inference engine is a machine that accepts Assumptions, Queries, and Data, and produces Testable implications, Estimand, and Estimate. The implications and estimand do not necessarily need to be presented at the end, but they are both implicit within the model. The first of the outputs is a Y/N decision as to whether the given query could be answered under the existing causal model, assuming perfect data. If the answer is Yes, the inference engine next produces an Estimand, a mathematical formula that generates the answer from any hypothetical data. If the answer is No, then the engine returns to the assumptions stage to refine the model, through either simplifying assumptions or adding new scientific knowledge. Finally, after the inference engine has received the Data, it will use the Estimand to produce an actual Estimate for the answer, along with the amount of uncertainty in that estimate.

Inference Engine

Approximate Inference affects the learning process which affects the accuracy of the algorithm. A behavior is seen in which approximating assumptions become self-fulfilling. Training the model with a unimodal approximate posterior will result in a posterior that is closer than would result from training with exact inference. Computing the harm by variational approximation is very difficult.

Inference Effects on Learning

Linear models are a good choice because they allow us to easily understand the relationship between X1, X2, X3.... and Y. These types of linear models are more restrictive and therefore easier to interpret. However, they do not allow us to consider statistical situations in which nonlinear relationships exist. 
Reference: James, G., Witten, D., Hastie, T., & Tibshirani, R. (2017). An introduction to statistical learning: with applications in R. New York: Springer.


Linear Models in Statistical Inference 

In statistical learning, inference typically aims to answer five key questions about the relationship between the predictors $$X_1, X_2, X_3, ..., X_P$$ and the outcome $$Y$$:

1. **Causal Relationship:** Is the relationship between the predictors and the outcome causal?
2. **Linearity:** Is the relationship linear or non-linear?
3. **Predictor Association:** Which predictors are associated with the outcome, and which are not?
4. **Direction:** What is the direction of the relationship for each predictor?
5. **Magnitude:** What is the magnitude of the relationship for each predictor?

Types of inferential questions in statistical learning

In statistical learning, inference means understanding the association of changes in the outcome $$Y$$ and changes in the predictors matrix $$X$$, i.e., estimating the characteristics of a function $$f$$ that maps the values of $$X$$ to corresponding values of $$Y$$.

University of Michigan - Ann Arbor

Google

Use cases of supervised statistical learning

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

A prediction problem in statistical learning occurs when there are a significant number of observations and their corresponding outcomes, and the goal is to predict outcomes for a new set of observations where they are unavailable. Let $$X$$ be the set of all input variables and $$Y$$ be the set of corresponding outcomes. If $$\hat{f}$$ is the estimate of the function $$f$$ that maps $$X$$ to $$Y$$, and $$\hat{Y}$$ is the resulting prediction of $$Y$$, assuming the error term averages to zero ($$\bar{\epsilon}=0$$), the following holds:

$$Y=f(X)+\epsilon, \bar{\epsilon}=0 \implies \hat{Y}=\hat{f}(X)$$

Learn Before

Related

Learn After