What is the correct formula for the penalty term in an elastic net regression?

Question

I've a question concerning the penalty term in an elastic net regression. In The elements of Statistical Learning by Hastie, Tibshirani & Friedman the formula (3.54) on p.73 says the penalty term is given by: $$ \lambda \cdot \displaystyle\sum_{j=1}^p (\alpha \cdot \beta_j^2 + (1 - \alpha) \cdot |\beta_j|) $$ This means for α=1 the formula transforms into a ridge regression. This is also consistent with the description in Zou & Hasti (2005, p. 303):

When α=1, the naive elastic net becomes simple ridge regression.

But almost every else (e.g., in the manual of the R package glmnet), it is written that ridge regression results if α=0. Actually, in the same book ("The elements...") on p. 681 in formula (18.20) it says, the penalty term has the form: $$ \displaystyle\sum_{j=1}^p (\alpha \cdot |\beta_j| + (1 - \alpha) \cdot \beta_j^2) $$

I wonder how this inconsistency can be explained. I would be grateful for any information.

Kind regards

Ulrich

Answer 1

I would not stress out over this. There isn't an established convention for representing how the elastic net penalty is parameterized (as evidenced by the different parameterizations within the same Elements of Statistical Learning text that you found), so you just need to be clear about how $\alpha$ is defined when discussing elastic net.

As a similar example, sometimes the LASSO objective function is expressed as $\sum_{i=1}^N (y_i - x_i^\top \beta)^2 + \lambda \|\beta\|_1$ while other times it is $\frac{1}{2}\sum_{i=1}^N (y_i - x_i^\top \beta)^2 + \lambda \|\beta\|_1$ or $\frac{1}{N}\sum_{i=1}^N (y_i - x_i^\top \beta)^2 + \lambda \|\beta\|_1$. Clearly the $\lambda$ in these different formulations are different, but it doesn't detract from discussion as long as everyone agrees on a definition beforehand.