Learn Before
Concept

analysis and advantage of elasticnet method

The elastic net method overcomes the limitations of the LASSO (least absolute shrinkage and selection operator) method which uses a penalty function based on L2 . Use of this penalty function has several limitations. For example, in the "large p, small n" case (high-dimensional data with few examples), the LASSO selects at most n variables before it saturates. Also if there is a group of highly correlated variables, then the LASSO tends to select one variable from a group and ignore the others. To overcome these limitations, the elastic net adds a quadratic part to the penalty , which when used alone is ridge regression (known also as Tikhonov regularization). The quadratic penalty term makes the loss function strongly convex, and it therefore has a unique minimum. The elastic net method includes the LASSO and ridge regression: in other words, each of them is a special case. Meanwhile, the naive version of elastic net method finds an estimator in a two-stage procedure : first for each fixed {\displaystyle \lambda _{2}}\lambda _{2} it finds the ridge regression coefficients, and then does a LASSO type shrinkage. This kind of estimation incurs a double amount of shrinkage, which leads to increased bias and poor predictions. To improve the prediction performance, the authors rescale the coefficients of the naive version of elastic net by multiplying the estimated coefficients by {\displaystyle (1+\lambda _{2})}(1 + \lambda_2).

Image 0

0

1

Updated 2021-02-07

Tags

Data Science