# Identity involving a projection matrix, originating from statistical regression theory

I am studying multiple linear regression and I am not able to understand a passage from my statistics textbook. The part that I do not understand can be formulated entirely in the language of linear algebra, as I will do here.

Let $$X \in \mathbb{R}^{n \times m}$$ an $$n \times m$$ matrix, with $$n \geq m$$ and with rank $$m$$. Let us denote $$x_1, \ldots, x_m$$ the $$m$$ column vectors. Let $$P$$ be the $$n\times n$$ matrix of the linear application describing the orthogonal projection from $$\mathbb{R}^n$$ onto the $$m-1$$-dimensional subspace $$\begin{equation*} \langle x_1, \ldots, x_{m-1}\rangle. \end{equation*}$$ Let $$I_n$$ be the $$n \times n$$ identity matrix. Let us denote with $$w^2_m$$ the value at position $$(m, m)$$ of the $$m \times m$$ square matrix $$(X^{\top}X)^{-1}$$. Then $$\begin{equation*} w^2_m = \frac{1}{x_m^{\top}(I_n - P)x_m}. \end{equation*}$$

Why does the last equation hold?

Probably it can be useful to know that $$P$$ can be written as $$\begin{equation*} P = X_0(X_0^{\top}X_0)^{-1}X_0^{\top}, \end{equation*}$$ where $$X$$ is the $$n \times m-1$$ matrix formed by the first $$m-1$$ columns of $$X$$.

Write $$X$$ as block matrix and calculate $$X^{\top}X$$: $$X=\begin{pmatrix}X_0 & x_m\end{pmatrix},$$ $$X^{\top}X=\begin{pmatrix}X^\top_0 \\ x^\top_m\end{pmatrix}\begin{pmatrix}X_0 & x_m\end{pmatrix}=\begin{pmatrix}X^\top_0X_0 & X^\top_0x_m\\ x^\top_mX_0 & x^\top_m x_m\end{pmatrix}.$$ To find the value $$w_m^2$$ at position $$(m,m)$$ of $$(X^{\top}X)^{-1}$$, we need to divide determinant of $$X^\top_0X_0$$ by the determinant of $$X^{\top}X$$: $$w_m^2=\frac{\det(X^\top_0X_0)}{\det(X^{\top}X)}.$$ With the help of determinant of block matrices https://en.wikipedia.org/wiki/Determinant#Block_matrices get $$\det(X^{\top}X) = \textrm{det}(X^\top_0X_0)\cdot \textrm{det}\bigl(x_m^\top x_m-x_m^\top \underbrace{X_0 (X_0^\top X_0)^{-1}X_0^\top}_{P} x_m\bigr)$$ $$=\det(X^\top_0X_0)\cdot \bigl(x_m^\top (I_n-P) x_m\bigr).$$ We skip last $$\det$$ since matrix $$x_m^\top (I_n-P) x_m$$ is $$1\times 1$$.
Finally, $$w_m^2=\frac{\det(X^\top_0X_0)}{\textrm{det}(X^{\top}X)} = \frac{1}{x_m^\top (I_n-P) x_m}.$$
• I do not understand why this is valid: "To find the value $w_m$ at position $(m,m)$ of $(X^{\top}X)^{-1}$, we need to divide determinant of $X^\top_0X_0$ by the determinant of $X^{\top}X$". Can you please develop it further? Thank you Dec 29, 2018 at 14:31
• Or look at en.wikipedia.org/wiki/… We need a cofactor to element $(m,m)$, and this is exactly the determinant of $X_0^\top X_0$.