3
$\begingroup$

Given an $n \times k$ matrix $A$ and an $n \times (n-k)$ matrix $B$ such that $\text{span}(A) \oplus \text{span}(B) = \mathbb{R}^n$, how to get the projection matrix on $\text{span}(A)$ parallel to $\text{span}(B)$, in terms of $A$ and $B$?


Edit:

Below it is worked out that $P=𝐴\Big(\big(𝐴^𝑇𝐴−𝐴^𝑇 𝐵(𝐵^𝑇 𝐵)^{−1}𝐵^𝑇 𝐴 \big)^{−1} 𝐴^𝑇 − (𝐴^𝑇 𝐴)^{−1}𝐴^𝑇 𝐵 \big(𝐵^𝑇𝐵−𝐵^𝑇𝐴(𝐴^𝑇𝐴)^{−1}𝐴^𝑇 𝐵 \big)^{-1}𝐵^𝑇 \Big)$.

  • Does it look right?
  • Can we possibly simplify it?
  • Is the horrendous look the reason that we rarely see it fleshed out in textbooks, etc.?

Edit 2:

I have verified numerically that the formula is correct. Also, with SVD $A = U_A \Sigma_A V_A^T$, $B = U_B \Sigma_B V_B^T$ and $U_{AB} = U_A^T U_B$, it can be simplified that $P = U_A \Big(I - U_{AB}^T U_{AB} \Big)^{-1} U_A^T - U_A U_{AB} \Big(I - U_{AB}^T U_{AB} \Big)^{-1}U_B^T$.

But I have little intuition on how to understand the expression. If anyone can provide some, that'd be great.

$\endgroup$
5
  • $\begingroup$ Hint: In a suitable basis, the matrix has the block form $\small{\left[\begin{array}{c|c}I&0\\ \hline 0&0\end{array}\right]}$. $\endgroup$
    – amd
    Mar 26, 2019 at 4:36
  • $\begingroup$ Let $C = [A|B]$ and let $c_j$ be the $j$-th column of $C$. Let $U$ be the matrix with $Uc_j = e_j$ and denote by $\bar U$ the $k\times n$ upper part of $U$. Then $P = A\bar U$. Indeed, $A\bar Ua_j = Ae_j = a_j$ and $A\bar Ub_i = A(0) = 0$. $\endgroup$
    – amsmath
    Mar 26, 2019 at 4:52
  • $\begingroup$ Can we express $P$ in terms of $A$ and $B$? (Question edited.) $\endgroup$
    – Lei
    Mar 26, 2019 at 4:57
  • $\begingroup$ Yes. See my answer below. $\endgroup$
    – amsmath
    Mar 26, 2019 at 5:05
  • $\begingroup$ There is an ordering error in the $U_{AB}$ terms in the first $(\ldots)^{-1}$ subexpression. The correct projector is $$P = U_A \Big(I - U_{AB} U_{AB}^T \Big)^{-1} U_A^T \;-\; U_A U_{AB} \Big(I - U_{AB}^T U_{AB} \Big)^{-1}U_B^T$$ $\endgroup$
    – greg
    Mar 28, 2021 at 15:41

1 Answer 1

3
$\begingroup$

Let $U = [A|B]^{-1}$ and denote by $\bar U$ the $k\times n$ upper part of $U$. Then $P = A\bar U$. Indeed, since $\bar U[A|B] = \left[\begin{matrix}I_k&0\end{matrix}\right]$, we have $A\bar Ua_j = Ae_j = a_j$ and $A\bar Ub_i = A(0) = 0$.

$\endgroup$
5
  • $\begingroup$ If we work out the expression, $P = A \Big( \big( A^T A - A^T B (B^T B)^{-1} B^T A \big)^{-1} A^T - (A^T A)^{-1} A^T B \big( B^T B - B^T A (A^T A)^{-1} A^T B \big)^{-1} B^T \Big)$. Does it look right? Can we possibly simplify it? Is the horrendous look the reason that we rarely see it fleshed out in textbooks, etc.? $\endgroup$
    – Lei
    Mar 26, 2019 at 5:13
  • $\begingroup$ @Lei Duuude, how did you get to this super ugly expression? I'm curious. $\endgroup$
    – amsmath
    Mar 26, 2019 at 5:18
  • $\begingroup$ I assumed $[A \; B]^{-1} = \big([A \; B]^T [A \; B]\big)^{-1} [A \; B]^T = \begin{bmatrix} A^T A \quad A^T B \\ B^T A \quad B^T B\end{bmatrix}^{-1} \begin{bmatrix} A^T \\ B^T \end{bmatrix}$. Then I used the formula for the inverse of block matrix. Is my first assumption correct? $\endgroup$
    – Lei
    Mar 26, 2019 at 5:33
  • 1
    $\begingroup$ Yes, it's correct because $(C^TC)^{-1}C^T = C^{-1}C^{-T}C^T = C^{-1}$. $\endgroup$
    – amsmath
    Mar 26, 2019 at 5:52
  • 1
    $\begingroup$ Your expression might look ugly, but when it comes to numerically computing $P$, then it is probably better because the sizes of the matrices that have to be inverted might be much smaller. Can you please check my answer? $\endgroup$
    – amsmath
    Mar 26, 2019 at 5:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .