Linear regression uniquely presents an optimization problem with a direct analytic solution. By subsuming the bias term into the weight vector $\mathbf{w}$—which is done by appending a column of all $1$s to the design matrix $\mathbf{X}$—the training objective becomes minimizing $\|\mathbf{y} - \mathbf{X}\mathbf{w}\|^2$. Setting the derivative of this loss with respect to $\mathbf{w}$ equal to $0$ yields the intermediate derivative formula $\partial_{\mathbf{w}} \|\mathbf{y} - \mathbf{X}\mathbf{w}\|^2 = 2 \mathbf{X}^ op (\mathbf{X} \mathbf{w} - \mathbf{y}) = 0$, which directly gives the normal equation $\mathbf{X}^ op \mathbf{y} = \mathbf{X}^ op \mathbf{X} \mathbf{w}$. Solving for $\mathbf{w}$ provides the optimal parameters: $\mathbf{w}^* = (\mathbf{X}^ op \mathbf{X})^{-1}\mathbf{X}^ op \mathbf{y}$. This closed-form solution exists only if the design matrix has full rank, ensuring that the matrix $\mathbf{X}^ op \mathbf{X}$ is invertible.