Assume we are working with a penalized linnear regression model. We have the following optimization problem:

\begin{equation} \min_{\beta}\left\{\left\lVert y-X\beta\right\rVert_2^2+\lambda\sum_{j=1}^p \left\vert \beta_j\right\vert\right\} \end{equation}

Equation above defines a linear regression model with a lasso penalization, while adaptive lasso penalization solves:

\begin{equation} \min_{\beta}\left\{\left\lVert y-X\beta\right\rVert_2^2+\lambda\sum_{j =1}^p w_j\left\vert \beta_j\right\vert\right\} \end{equation}

where $w$ defines a vector of weights that are fixed, so the only variable in the equation is $\beta$.

This adaptive idea was initially proposed in "The adaptive Lasso and its Oracle Properties" (Journal of the American Statistical Association 101.476 (2006): 1418-1429.), and in this paper, in section 3.5, the authors state that it is possible to solve the adaptive lasso penalization using any algorithm for solving lasso penalization. For doing so, we need to do the following steps:

  1. Define $x_j^{**}=x_j/\hat{w_j}, j=1,\ldots,p$
  2. Solve the lasso problem \begin{equation} \hat{\beta}^{**}=argmin_{\beta}\left\{\left\lVert y- \sum_{j=1}^px_j^{**}\beta_j\right\rVert^2+\lambda\sum_{j=1}^p\left\vert \beta_j\right\vert\right\} \end{equation}
  3. Recover the adaptive lasso estimator as $\hat{\beta_j}^*=\hat{\beta}_j^{**}/w_j$

I am trying to prove that these steps are mathematically correct but I found myself unable to do so, I would appreciate any hint or explanation on this.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.