1
$\begingroup$

The Woodbury matrix identity is defined as follows: $$ {(A+UCV)}^{-1}=A^{-1}-A^{-1}U{(C^{-1}+VA^{-1}U)}^{-1}VA^{-1} $$

I want to use the Woodbury matrix identity theorem to change the following matrix formula $$ W={(XX^T+\lambda G)}^{-1}XY $$ into the following form $$ W=G^{-1} X {(X^TG^{-1}X+\lambda I)}^{-1}Y $$ The dimensions are as follows: $$ X\in R^{p\times n}\\ G\in R^{p\times p}\\ Y\in R^{n\times c} $$ Could anyone help give some hints?

UPDATE:

From the two formulas about $W$, we could get the following equations, thus the two $W$s should be equal: enter image description here

$\endgroup$
6
  • $\begingroup$ It might help to give dimensions? $\endgroup$
    – copper.hat
    Dec 29, 2014 at 7:22
  • $\begingroup$ Hi, @copper.hat, I've updated the post. Please check it. Thanks! $\endgroup$
    – mining
    Dec 29, 2014 at 7:27
  • $\begingroup$ Are you sure you stated the identity correctly? The link has something different. (Missing parantheses on the left hand side) $\endgroup$
    – Max
    Dec 29, 2014 at 7:33
  • $\begingroup$ Hi, @Max, thank you, I've revised the mistake. $\endgroup$
    – mining
    Dec 29, 2014 at 7:36
  • 1
    $\begingroup$ @Max, I've found the solution as posted in the answer. If you were interested, please check it. Thank you! $\endgroup$
    – mining
    Dec 30, 2014 at 12:23

2 Answers 2

Reset to default
2
$\begingroup$

I've found the solution as follows:

My Derivation

$\endgroup$
2
  • $\begingroup$ Yes, that looks correct to me. $\endgroup$
    – ki3i
    Dec 30, 2014 at 17:46
  • $\begingroup$ Wow, nice job ! $\endgroup$
    – Max
    Dec 30, 2014 at 20:38
1
$\begingroup$

I'm not sure the different forms of W, as stated, are equivalent. For one thing, they do not appear to be equivalent when the matrices involved are replaced by scalars. To illustrate, let $X=a$, $G=b$ and $Y=c$. Then, $$ \begin{eqnarray*} W{}={}{(XX^T+\lambda G)}^{-1}XY &{}\implies{}&W{}={}\frac{ac}{a^2{}+{}\lambda b}\,,\newline \end{eqnarray*} $$ while $$ \begin{eqnarray*} W{}={}G^{-1} X {(X^TG^{-1}+\lambda I)}^{-1}Y &{}\implies{}&W{}={}\frac{ac}{a{}+{}\lambda b}\,.\newline \end{eqnarray*} $$

Furthermore, direct manipulation of the first posted equation with $W$ gives $$ W{}={}G^{-1} X {(X^TG^{-1}X+\lambda I)}^{-1}Y\,, $$ which is different from the second $W$ equation posted but, now, seems consistent (assuming, in addition, that $X$ is invertible).

$\endgroup$
5
  • $\begingroup$ Hi, @ki3i, I'm very sorry that I've missed one $X$ in the second formula of $W$. I just found that fault. $\endgroup$
    – mining
    Dec 29, 2014 at 10:47
  • $\begingroup$ @hubberwinston , I think you might have to verify the formulae you are interested in, carefully. In my derivation of $W$'s alternative form, I assumed that $X$ was invertible; this is clearly not the case, based on $X$ not being a square matrix. So, I suspect the $W$ equations, as stated now, are still inconsistent. $\endgroup$
    – ki3i
    Dec 29, 2014 at 12:39
  • $\begingroup$ thank you for your help! I've updated the post. Please check it. The $W$ equations should be consistent. But I still couldn't find the direct solution from the first $W$ to the second $W$. $\endgroup$
    – mining
    Dec 29, 2014 at 14:59
  • $\begingroup$ @hubberwinston , The argument you show, starting from ${(XX^T+\lambda G)}^{-1}X {}={} G^{-1} X {(X^TG^{-1}X+\lambda I)}^{-1}$, does not follow from ${(XX^T+\lambda G)}^{-1}XY {}={}W{}={} G^{-1} X {(X^TG^{-1}X+\lambda I)}^{-1}Y$ since the matrices multiplying $Y$ on either side are not necessarily invertible square matrices. I don't think the stated expressions for $W$ are equivalent without extra assumptions e.g. invertibility of $X$. $\endgroup$
    – ki3i
    Dec 29, 2014 at 21:42
  • $\begingroup$ Hi, @ki3i, I've found the solution as posted in the answer. If you were interested, please check it. Thank you! $\endgroup$
    – mining
    Dec 30, 2014 at 12:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .