I want to show $$GM^\top(MGM^\top)^{-1}MG=G$$ where $MGM^\top$ is invertible, $G$ is a symmetric square matrix. $M$ does not have to be a square matrix. I am not 100% sure this is true (90% sure it is) so a contradictory proof will be great as well.

Thanks Justingpassby for the contradictory proof. I came into above from the following but I'm not sure where it is wrong?

Let $u = Mv$, $\text{var}(v)=G$, $u$ and $v$ be both Normal random variables with mean 0 so $\text{var}(u)=MGM^\top$. Then $v=GM^\top(MGM^\top)^{-1}u$ is one possible solution of $u = Mv$ since $$ Mv=MGM^\top(MGM^\top)^{-1}u = u.$$ Now $$\begin{array}\text{var}(v) &=& GM^\top(MGM^\top)^{-1}MGM^\top(MGM^\top)^{-1}MG\\ &=& GM^\top(MGM^\top)^{-1}MG \end{array}$$ Therefore $GM^\top(MGM^\top)^{-1}MG=G$. But as shown this it not true for some non-square $M$ so there must be a hole in my logic somewhere above. Any idea anyone?

  • $\begingroup$ I do not understand the first line in the expression for $var(v).$ $\endgroup$ – Justpassingby Dec 7 '15 at 0:39
  • $\begingroup$ It is variance of $v$ but fair enough, this bit is statistical. $\endgroup$ – aimi Dec 7 '15 at 0:43
  • $\begingroup$ But where does it come from? It does not seem to follow the same rule as your earlier computation of $var(u).$ $\endgroup$ – Justpassingby Dec 7 '15 at 0:45
  • $\begingroup$ Hi, we let $var(v) = G$. Then since $u=Mv$ and it is normal rv, it follows that $var(u) = MGM^\top$. $\endgroup$ – aimi Dec 7 '15 at 1:21
  • 1
    $\begingroup$ Might be worth noting that the rest of the expression after the first $G$ on the lhs gives the orthogonal projection onto the row space of $M$ relative to the scalar product defined by $G$. $\endgroup$ – amd Dec 7 '15 at 1:28

Let $G$ be the unit 3 by 3 matrix and $M$ any nonzero 1 by 3 row. Then the LHS has rank 1 (and determinant 0).

  • $\begingroup$ Thanks, elegant contradictory proof. I just added my logic of how I got to this but I'm not sure where it fell short. Will you be enlighten me where it might have gone wrong? $\endgroup$ – aimi Dec 7 '15 at 0:28
  • $\begingroup$ I would like to understand where the equality $\text{var}(v)=GM^\top(MGM^\top)^{-1}MGM^\top(MGM^\top)^{-1}MG$ comes from. It gives you the variance of a possible $v$ that solves $u=Mv$ but there is no guarantee that it is the variance of the original $v$ that you started with. $\endgroup$ – Justpassingby Dec 9 '15 at 7:22
  • $\begingroup$ Actually you are correct. That is probably where my logic failed. Sorry must have been staring at the problem too long to not realise that obvious -.- $\endgroup$ – aimi Dec 9 '15 at 8:56

If $L$ is the left side, $$M L M^T = (MGM^T)(MGM^T)^{-1}(MGM^T) = MGM^T$$ If $M$ is a square matrix, $M$ must be invertible for $MGM^T$ to be invertible, so $L = G$. If $M$ is not square, then as Justpassingby noted $L$ does not have full rank, whereas $G$ could have full rank.



$$(MGM^\top)^{-1}= M^{\top(-1)}G^{-1}M^{-1}$$


$$\begin{align} GM^\top(MGM^\top)^{-1}MG &= GM^\top M^{\top(-1)}G^{-1}M^{-1} MG \\ & = G \end{align}$$

since everything else cancels.

This assumes that both $G$ and $M$ are invertible.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.