This is kind of embarrassing; I once knew this stuff, and I've forgotten it. I've got a fitted ridge regression:

$$ \hat\beta = \left(X'X+\lambda\right)^{-1}X'y $$ X is n by k

y is n by 1

$\lambda$ is k by k

$\hat\beta$ is k by 1

The inverted matrix is invertible.

How do I solve for lambda? I've got the data $X,y$ and the parameters $\hat\beta$

Edit: potentially useful bit of information: ridge penalties are usually a vector multiplied by an identity matrix. But in this problem I can't assume that the off-diagonals are zero.

Edit2: So, the answer should have been obvious. A fitted model implies an estimated $V_p$ matrix. Divide by estimated dispersion parameter and any degrees of freedom correction, and you've got $(X'X+\lambda)^{-1}$. invert and get $\lambda$.


I assume that $\lambda$ is a diagonal matrix.

$$ \lambda \hat\beta + X'X\hat\beta = X'y $$

You get: $$ \lambda_i= \frac1{ \hat\beta_i} \left\{\sum_k X_{ki}y_k - \sum_{k,l} X_{ki}X_{kl}\hat\beta_l\right\} $$as long as $\hat\beta_i\neq 0$.

  • $\begingroup$ I got as much with pencil and paper. But how do I isolate lambda? $\endgroup$ – JPTP Apr 4 '14 at 2:09
  • $\begingroup$ look at one coordinate $\hat\beta_i\neq 0$. $\endgroup$ – mookid Apr 4 '14 at 2:10
  • $\begingroup$ what do you mean by coordinate? $\endgroup$ – JPTP Apr 4 '14 at 2:11
  • $\begingroup$ one entry of the vector $\hat \beta$. $\endgroup$ – mookid Apr 4 '14 at 2:12
  • $\begingroup$ Now I'm confused in a different way. Say $\hat\beta_1$ = 2. This would give me $\lambda = X'y*.5-X'X$. What does that even mean? $\hat\beta$ is a vector, and that gives me a matrix for one single beta. I think my problem is that I forget something relatively basic in matrix algebra, and it remains an unknown unknown. $\endgroup$ – JPTP Apr 4 '14 at 2:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.