Consider a matrix $A = \begin{bmatrix} A_1 & A_2 \end{bmatrix}\in \mathbb{R}^{n\times(d_1+d_2)}$ with $rank(A) = d_1+d_2=d\leq n$. That is, the columns of $A$ are linearly independent. I want to represent $P_{R(A)}^\perp$, the projection onto the orthogonal complement of $R(A)$ (the range or image of $A$) as the following $$ P_{R(A)}^\perp = P_{R(A_1)}^\perp P_{R(A_2)}^\perp. $$

We have that $R(A)= R(A_1)+R(A_2)$ so $R(A)^\perp = R(A_1)^\perp\cap R(A_2)^\perp$. However, in order to have the representation to hold I have read that we need to establish that $$ P_{R(A_1)}^\perp P_{R(A_2)}^\perp = P_{R(A_2)}^\perp P_{R(A_1)}^\perp, $$ which I can't seem to prove. This is equivalent with showing that

$$ P_{R(A)} = P_{R(A_1)} + P_{R(A_2)}- P_{R(A_2)}P_{R(A_1)}. $$

Do I need further assumptions or can the representation be shown to hold under the assumption of linearly independent columns of $A$?


Consider $$A = \left[\begin{array}{c|c}1 & 1\\0 & 1\\0&0\end{array}\right]$$

Then $R(A) = \{\begin{bmatrix}x&y&0\end{bmatrix}^T\mid x,y \in \Bbb R\}$ and $R(A)^\perp = \{\begin{bmatrix}0&0&z\end{bmatrix}^T\mid z \in \Bbb R\}$ so $$P_{R(A)}^\perp = \begin{bmatrix}0&0&0\\0&0&0\\0&0&1\end{bmatrix}$$ $R(A_1) = \{\begin{bmatrix}x&0&0\end{bmatrix}^T\mid x \in \Bbb R\}$ and $R(A_1)^\perp = \{\begin{bmatrix}0&y&z\end{bmatrix}^T\mid y,z \in \Bbb R\}$ so $$P_{R(A_1)}^\perp = \begin{bmatrix}0&0&0\\0&1&0\\0&0&1\end{bmatrix}$$ And $R(A_2) = \{\begin{bmatrix}t&t&0\end{bmatrix}^T\mid t \in \Bbb R\}$ and $R(A_2)^\perp = \{\begin{bmatrix}t&-t&z\end{bmatrix}^T\mid y,z \in \Bbb R\}$ so $$P_{R(A_2)}^\perp = \begin{bmatrix}\frac12&-\frac12&0\\-\frac12&\frac12&0\\0&0&1\end{bmatrix}$$ Obviously $$P_{R(A_2)}^\perp P_{R(A_1)}^\perp \ne P_{R(A)}^\perp \ne P_{R(A_1)}^\perp P_{R(A_2)}^\perp$$ In order for this to work, $R(A_1)$ must be orthogonal to $R(A_2)$.

  • $\begingroup$ Thank you for the example. Are you sure that orthogonality is necessary? $\endgroup$
    – John
    Mar 23 '20 at 14:17
  • 1
    $\begingroup$ Yes. What I did here was grab the simplest example where $R(A_1)$ was not orthogonal to $R(A_2)$. If subspaces $V,W$ are not perpendicular, then for some $v \in V, P_W(v)$ will not be $0$. If you consider the plane of $v, P_W(v)$, you can see geometrically that projecting $P_W(v)$ back onto $v$ results in a shorter, but still non-zero vector. So $P_WP_V(v) = P_W(v)$ while $P_VP_W(v) = av$ for some $a < 1$. $R(A_1)^{\perp}$ is orthogonal to $R(A_2)^{\perp}$ exactly when $R(A_1) \perp R(A_2)$. $\endgroup$ Mar 23 '20 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.