2
$\begingroup$

Suppose that I have a $n \times k$ real matrix with full column rank. Say $k=3$ and I write

$$X = [\mathbf x_1:\mathbf x_2:\mathbf x_3]$$

where lower-case $\mathbf x$'s are $n \times 1$ vectors.

I go on and form the orthogonal projection matrix

$$P_X = X \left(X'X\right)^{-1}X'$$.

Consider now the matrix

$$W = [\mathbf x_1:-\mathbf x_2:\mathbf x_3]$$

Namely it is equal to $X$ matrix, but in (any) one column, the sign of the elements are switched.

Question: Can we express the projection matrix of $W$, $P_W=W \left(W'W\right)^{-1}W'$, in terms of the projection matrix of $X$, $P_X$, or at least state some relation between them?

I tried to explore this with what little matrix algebra I know, but could not come up with anything. In reality the $k$ dimension is bigger that $3$ but I guess this does not matter.

$\endgroup$
3
$\begingroup$

We have $P_X = P_W$.

We can show this using matrix algebra by noting that $W = XQ$, where $Q$ is the orthogonal matrix $$ Q = \pmatrix{1&0&0\\0&-1&0\\0&0&1}. $$ With that, we note that $$ \begin{align} P_W &= W(W'W)^{-1}W' = [XQ]([XQ]'[XQ])^{-1}[XQ]' \\ & = XQ[Q' (X'X) Q]^{-1}Q'X \\ & = XQ[Q'(X'X)^{-1}Q]Q'X \\ & = X[QQ'](X'X)^{-1}[QQ']X = X(X'X)^{-1}X = P_X, \end{align} $$ as was desired.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This proof can be extended (with judicious uses of $Q^{-1}$ rather than $Q'$) to the case where $Q$ is an arbitrary invertible matrix. $\endgroup$ – Omnomnomnom May 30 at 17:31
  • $\begingroup$ You do not need computations. We are talking about the same subspace, so the matrix in a fixed basis has to be the same. $\endgroup$ – GReyes May 30 at 21:46
  • $\begingroup$ @G I took this approach because the asker mentioned trying to explore this with “matrix algebra” $\endgroup$ – Omnomnomnom May 31 at 16:17
2
$\begingroup$

The reason why the matrices are the same is because you are projecting on the same subspace. There is no need for a proof here. Even if you replace your columns by some linear combinations that have the same span, you get the same matrix. They do not have to be independent either.

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

The projection is defined uniquely by the properties that it is the identity map on the column space of $X$ and zero on the orthogonal complement. The matrix that represents a linear transformation in a given basis (here, the standard basis) is unique, so if you use any other full-rank matrix $Y$ that has the same column space as $X$, you’ll end up with the same projection matrix.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.