In ridge regression, the model coefficients and bias are learned using the least-squares criterion with an added penalty for large coefficients. The parameters are found by minimizing an objective function that includes a tuning parameter (often denoted as $\lambda$ or $\alpha$) to control the strength of this penalty term. Once the parameters are learned, the prediction formula is the same as in ordinary least squares (OLS).

Ridge regression uses $L_2$ regularization (weight decay), which adds the sum of the squared coefficients to the residual sum of squares (RSS):

$RSS_{RIDGE}(\beta_{j}, \beta_{0}) = \sum_{i=1}^{n} ({y}_{i}-{\beta}_{0}-\sum_{j=1}^{p}{\beta}_{j}{x}_{ij})^2 +\lambda\sum_{j=1}^{p}{\beta}_{j}^2=RSS+\lambda\sum_{j=1}^{p}{\beta}_{j}^2$

A higher tuning parameter ($\lambda$ or $\alpha$) results in more regularization and a simpler model. Ridge regression is especially useful when the number of predictor variables is greater than the number of observations. Because the penalty term is sensitive to the scale of the variables, predictors should usually be standardized before applying ridge regression.