# A matrix identity

Suppose $$K^TX=0$$ and $$K$$ has full column rank. Suppose furthermore that the columns of $$X$$ form a basis for the nullspace of $$K^T$$. If $$H$$ is positive definite, then

$$K(K^THK)^{-1}K^T = H^{-1} − H^{-1}X(X^TH^{-1}X)^{-1}X^TH^{-1}$$

The identity can be found on slide 44 at http://www.ltcc.ac.uk/media/qmul-images/REML-Lecture-1.pdf. A quick R script has also verified the formula. However, I can't seem to prove the identity. I have tried using the Schur formula for inverses, but it does not seem to help in this case.

• You need to specify that the columns of $X$ give a basis for the nullspace of $K^\top$. Then you should see why the formula holds when $H$ is the identity. Dec 16, 2022 at 5:22
• @TedShifrin Thank you Prof. Shifrin. That makes perfect sense now: In the case where $H=I$, then for any vector $v$ we may express $v$ uniquely as $v=K \gamma_1 + X\gamma_2$. Then, $K(K^TK)^{-1}K^T(K \gamma_1 + X\gamma_2) + X(X^TX)^{-1}X^T(K \gamma_1 + X\gamma_2) = K \gamma_1 + X\gamma_2$ is straightforward to see which yields the formula $K(K^TK)^{-1}K^T + X(X^TX)^{-1}X^T = I$. When $H \neq I$, if we could somehow express $v=HK \gamma_1 + X \gamma_2$, we could then use the same strategy to show the formula holds - but it is not obvious to me that $v=HK \gamma_1 + X \gamma_2$
– Abm
Dec 16, 2022 at 7:46

Although you have not revised appropriately, we agree that the columns of $$X$$ give a basis for the nullspace of $$K^\top$$. This subspace is the orthogonal complement of the column space $$W$$ of $$K$$. Assuming $$H=I$$, we therefore have \begin{align*} I &= \text{proj}_W + \text{proj}_{W^\perp} \\ &= K(K^\top K)^{-1}K^\top + X(X^\top X)^{-1}X^\top. \end{align*} Here $$\text{proj}_V$$ is the matrix for the orthogonal projection onto the subspace $$V$$.
In the general case, we let $$\tilde K = \sqrt H K$$, where $$\sqrt H$$ is the unique positive square root of the positive definite symmetric matrix $$H$$. We set $$\tilde X = \sqrt H^{-1} X$$. Then the columns of $$\tilde X$$ give a basis for the nullspace of $$\tilde K$$. Thus, as before, we have \begin{align*} I &= \tilde K(\tilde K^\top\tilde K)^{-1}\tilde K^\top + \tilde X(\tilde X^\top\tilde X)^{-1}\tilde X^\top \\ &= \sqrt H K(K^\top H K)K^\top \sqrt H + \sqrt H^{-1}X^\top(X^\top H^{-1}X)^{-1}X^\top\sqrt H^{-1}, \text{so} \\ H^{-1} &= K(K^\top HK)K^\top + H^{-1}X^\top(X^\top H^{-1}X)^{-1}X^\top H^{-1}, \end{align*} as desired.
• Thank you again, Prof. Shifrin, fully understood. I did not think of forming the matrices $\bar{K}$ and $\bar{X}$ - that was nice. I will update my question to include the correct assumptions on $X$ for future readers