The ridge estimator can be written in the following way, where the singular value decomposition of X is $X=UDV^{'}$.

enter image description here

I can't quite figure out how the last step (4th step) was obtained from the 3rd one. I think it is something about matrix inverse. Can anyone help me? Thank you.

  • $\begingroup$ $(ABC)^{-1}=C^{-1}B^{-1}A^{-1}$ $\endgroup$ – AnyAD Jun 15 '18 at 14:57
  • $\begingroup$ HI, i know that rule. But can i use that when there is a sum of matrices as well ? $\endgroup$ – student_R123 Jun 15 '18 at 15:04
  • $\begingroup$ There is no sum, $B=D^2+\lambda I $ $\endgroup$ – AnyAD Jun 15 '18 at 15:06
  • $\begingroup$ $A=V $ and $C=V^T$ $\endgroup$ – AnyAD Jun 15 '18 at 15:07
  • $\begingroup$ I see. Then i have to use the orthogonal property of matrix V to obtain the desired result. isn't it ? $\endgroup$ – student_R123 Jun 15 '18 at 15:13

Since $V$ (and $U$) is a unitary matrix, i.e., $VV^T = I \Leftrightarrow V^{-T} = V$, and $V^TV = I \Leftrightarrow V^T = V^{-1}$, \begin{align} \beta(\lambda) &=\left(VD^2V^T + \lambda V V^T\right)^{-1} VDU^TY \\ &= \left(V\left(D^2 + \lambda I \right)V^T\right)^{-1} VDU^TY \\ &= V^{-T}\left(D^2 + \lambda I \right)^{-1} V^{-1} VDU^TY \\ &= V\left(D^2 + \lambda I \right)^{-1} V^T VDU^TY \\ \end{align}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.