# Question about projection onto subspaces

Suppose I have an $$n\times n$$ (complex) matrix $$Q$$ viewed as a linear transformation from $$\mathbb{C}^{n}$$ to $$\mathbb{C}^{n}$$. I'm struggling to understand the difference between the following two scenarios. For simplicity, let us consider $$n = 4$$.

Scenario 1: Let $$S$$ be a subspace of $$\mathbb{C}^{4}$$ of dimension $$d=2$$. Let us think of $$S$$ as the subspace generated by two eigenvectors $$v_{1}$$ and $$v_{2}$$ of $$Q$$.One can construct a projection matrix $$P_{S}$$. According to this post, this projection should be given by: $$P_{S} = A(A^{T}A)^{-1}A^{T}$$ where $$A = [v_{1} \hspace{0.1cm} v_{2}]$$ the a $$4\times 2$$ matrix. This is a $$4\times 4$$ matrix.

Scenario 2: Let us write $$\mathbb{C}^{4} = S \oplus S^{\perp}$$. Since $$Q$$ is a linear operator on $$\mathbb{C}^{4}$$, it might be possible to decompose $$Q = Q_{1}\oplus Q_{2}$$ so that, if $$u = u_{1}+u_{2}$$ and $$u_{1}\in S$$ and $$u_{2} \in S^{\perp}$$: $$Qu = Q_{1}u_{1}\oplus Q_{2}u_{2}$$ Each $$Q_{i}$$ should be a $$2\times 2$$ matrix.

Here is my problem. I would expect that $$Q_{1} = QP_{S}$$, since $$Q_{1}$$ should be the restriction of $$Q$$ to $$S$$. However, as I stressed before, both $$Q$$ and $$P_{S}$$ are $$4\times 4$$ matrices. So, where is the gap? If $$Q_{1}$$ is not $$QP_{S}$$, how can I obtain $$Q_{1}$$? And finally, why is $$P_{S}$$ a $$4\times 4$$ matrix if it projects into a subspace of dimension $$2$$? (I would expect it to be a $$2\times 4$$ matrix instead).

If $$V$$ is a finite dimensional inner product space (like $$\mathbb{C}^{n}$$) and $$S$$ is a subspace, we have two similar notions. There is the orthogonal projection operator $$P_{S}:V\rightarrow V$$ that intuitively takes $$x\in V$$ to the nearest element of the subspace $$S$$, but it is still a mapping $$V\rightarrow V$$. This operator satisfies $$P^{2} = P$$, $$P = P^{*}$$, and $$\text{Im}(P)=S\subset V$$, and it happens that $$I-P_{S} = P_{S^{\perp}}$$. There is also the decomposition $$V = S\oplus S^{\perp}$$ and the mappings $$\pi_{S}:V\rightarrow S$$ and $$\pi_{S^{\perp}}:V\rightarrow S^{\perp}$$ that map $$v\in V$$ to $$P_{S}(v)\in S$$ or $$P_{S^{\perp}}v\in S^{\perp}$$ respectively. Let's suppose that $$V$$ has dimension $$n$$ and $$S$$ has dimension $$k$$. As functions, $$P_{S}$$ and $$\pi_{S}$$ map $$x$$ to the same element, but they have different codomains $$V$$ and $$S$$ respectively. When we write $$P_{S}$$ and $$\pi_{S}$$ as matrices, they are $$n\times n$$ and $$k\times n$$ respectively.

In a moment, we will need the inclusion mappings $$i_{S}:S\rightarrow V$$ and $$i_{S^{\perp}}:S^{\perp}\rightarrow V$$ that simply map a vector in the subspace to the same vector in the larger vector space $$V$$. It may be helpful to observe that even though these mappings essentially "do nothing", they are represented by rectangular (nonsquare) matrices.

Given a linear operator $$Q$$, we cannot generally decompose it as $$Qu = Q_{1}u\oplus Q_{2}u$$ like you wrote in your post (unless you interpret the notation $$\oplus$$ strangely). We can decompose it into a block matrix with four blocks. We should choose a basis $$V$$ of the form $$s_{1},\ldots,s_{k},q_{1},\ldots,q_{n-k}$$ so that $$s_{1},\ldots,s_{k}$$ is a basis of $$S$$ and $$q_{1},\ldots,q_{n-k}$$ is a basis of $$S^{\perp}$$. Then

$$Q = \begin{pmatrix}\pi_{S}Qi_{S} & \pi_{S}Qi_{S^{\perp}}\\ \pi_{S^{\perp}}Qi_{S} & \pi_{S^{\perp}}Qi_{S^{\perp}}\end{pmatrix}.$$

If $$V$$ is $$n$$-dimensional and $$S$$ is $$k$$ dimensional, then we have cut up the $$n\times n$$ matrix $$Q$$ into (clockwise) an $$k\times k$$, $$k\times (n-k), (n-k)\times (n-k)$$ and $$(n-k)\times k$$ submatrices.

We may also write it in a very similar way as a sum of four operators (square matrices mapping V to itself). Remember that $$P_{S^{\perp}} = I - P_{S}$$. Then $$Q = IQI = (P_{S} + I-P_{S})Q(P_{S}+I-P_{S}) = P_{S}QP_{S} + P_{S}QP_{S^{\perp}} + P_{S^{\perp}}QP_{S} + P_{S^{\perp}}QP_{S^{\perp}}$$.

To give an example to relate these concepts, observe that

$$P_{S}QP_{S^{\perp}} = \begin{pmatrix}0 & \pi_{S}Qi_{S^{\perp}}\\0 & 0\end{pmatrix}.$$

Finally, I will try to answer your problem concretely. You wrote "$$Qu = Q_{1}u \oplus Q_{2}u$$" where $$\oplus$$ refers to the decomposition $$V = S\oplus S^{\perp}$$. I think this is an improper use of this notation, since if we write $$x = y\oplus z$$, it should be that $$y\in S$$ and $$z\in S^{\perp}$$, but $$Q_{1}$$ and $$Q_{2}$$ will not necessarily produce values in $$S$$ and $$S^{\perp}$$ respectively. [If they do, then the decompositions above are block diagonal, which is an important special case.] The closest thing to $$Q_{1}$$ from your post would be $$Qi_{S}$$, which takes vectors from $$S$$ to $$V$$ and is, therefore, $$4\times 2$$ (a mapping from $$\mathbb{C}^{2}\rightarrow \mathbb{C}^{4}$$). Compare the block decompositions of $$QP_{S}$$ and $$Qi_{S}$$ below:

$$QP_{S} = \begin{pmatrix}\pi_{S}Qi_{S} & 0\\ \pi_{S^{\perp}}Qi_{S} & 0\end{pmatrix}\in \mathbb{C}^{n\times n},$$ and $$Qi_{S} = \begin{pmatrix}\pi_{S}Qi_{S}\\\pi_{S^{\perp}}Qi_{S}\end{pmatrix}\in \mathbb{C}^{k\times n}.$$

You also ask "why is $$P$$ a $$4\times 4$$ mapping if it projects to a subspace of dimension two?" Projections refer to $$P_{S}$$ not $$\pi_{S}$$. I think you are confusing these two mappings. They are essentially the same, except that they have different codomains. That makes them very different when you represent them as matrices.

• That's a very nice answer, thanks! As for $P_{S} = A(A^{T}A)^{-1}A^{T}$, are there formulas where we can express both $\psi_{S}$ and $i_{S}$ as matrices? At least in my case, where $S$ is generated by two vectors $v_{1}$ and $v_{2}$. Jul 23 at 22:54
• In order to even start writing $A$ or $P_{S}$ as a matrix, we need to choose a basis of $V$ (in $V$ is $\mathbb{C}^{4}$, then the basis is $e_{1},e_{2},e_{3},e_{4}$). In order to write mappings $V\rightarrow S$ like $\pi_{S}$, we need to chose a basis for both $V$ and $S$. Even if $V$ is $\mathbb{C}^{4}$ so that the basis is obvious, $S$ could be any weirdo subspace of $\mathbb{C}^{4}$, so we will need to pick a basis of it somehow to write $\pi_{S}$. There is no obvious way/natural way to do this, so it's hard to write a formula analogous to $P_{S} = A(A^{T}A)^{-1}A^{T}$. Jul 23 at 23:04
• This issue doesn't happen when we are writing matrices for operators $\mathbb{C}^{4}\rightarrow \mathbb{C}^{4}$, because we only need the one obvious basis. But suppose that we pick an orthogonal basis of $V$ like in the answer: $s_{1},s_{2},v_{1},v_{2}$ so that $S$ is the span of $s_{1}$ and $s_{2}$. Then $i_{S}$ is $\begin{pmatrix}1 & 0\\ 0 & 1 \\ 0 & 0\\ 0 & 0\end{pmatrix}$ and $\pi_{S}$ is the transpose. [In general, $\pi_{S}$ and $i_{S}$ are adjoints/conjugate transposes of one another.] Jul 23 at 23:07
• Right. I'm assuming you are assuming $Q$ is diagonal here, right? Jul 23 at 23:40
• None of my comments make any reference to $Q$, and my answer does not assume that $Q$ is diagonal. Jul 24 at 0:05

I started working on a trivial example before the much better answer appeared, but perhaps this simple concrete example will help guide someone.

You can express tensors in different bases. Just like I can express the 2D vector at $$v = 3 \mathbf{e}_x + 4 \mathbf{e}_y$$ in terms of basis vectors for an $$x-y$$-plane, I can also express it in terms of $$v = 5 \mathbf{e}_v$$ where $$\mathbf{e}_v = \frac{3}{5} \mathbf{e}_x + \frac{4}{5} \mathbf{e}_y$$.

When we only pass around matrices with just the coefficients, e.g. $$v = [3,4]$$ there is an assumption made of what base they are expressed in. If we want to project another vector, e.g. $$u = 5 \mathbf{e}_x + 0 \mathbf{e}_y$$ we can express this projection in several ways, in terms of the 2 coefficients of $$\mathbf{e}_x$$ and $$\mathbf{e}_y$$, or in terms of the 1 coefficient of $$\mathbf{e}_v$$. Working through the computations for the case I made up here, one will get $$u_v = \frac{9}{5} \mathbf{e}_x + \frac{12}{5} \mathbf{e}_y \equiv 3 \mathbf{e}_v$$ Either way to tackle it, it's the same vector, we only need to think of what base we are expressing the coefficients in.

There is something fiddly happening here which basically boils down to the fact that you can view $$Q_1$$ (and $$Q_2$$) as matrices in $$GL(2;\mathbb{C})$$ since they act on a $$2$$ dimensional space, or view them as elements of $$GL(2;\mathbb{C}) \subset GL(4;\mathbb{C})$$ i.e. view them as $$4 \times 4$$ matrices.

As $$S$$ is viewed as a subset of $$\mathbb{C}^4$$ it is a good idea to view $$u_1$$ and $$u_2$$ as vectors with four entries even though $$u_1 \in S$$ and $$u_2 \in S^{\perp}$$, which are both $$2$$ dimensional subspaces of our original space.

As in example, if our space is $$\mathbb{C}^4$$ and we consider $$S = \{e_1,e_2\}$$ where $$e_1$$ and $$e_2$$ are the standard basis vectors then we would still want to write, $$e_1 = \begin{pmatrix} 1 \\0\\0\\0 \end{pmatrix}$$ rather than $$\begin{pmatrix} 1 \\ 0 \end{pmatrix}$$ even though $$e_1 \in S$$, where $$S$$ is a $$2$$-dimensional space, due to the fact that we view $$S$$ as a subspace of $$\mathbb{C}^4$$.

Thus, we need $$Q_1$$ and $$Q_2$$ to be $$4 \times 4$$ matrices, it might look something like this:

$$Q = \begin{pmatrix} 1&0&0&0\\0&4&0&0\\0&0&5&6\\0&0&7&8 \end{pmatrix} = \begin{pmatrix} 1&0&0&0\\0&4&0&0\\0&0&0&0\\0&0&0&0 \end{pmatrix} + \begin{pmatrix} 0&0&0&0\\0&0&0&0 \\0&0&5&6\\0&0&7&8 \end{pmatrix} = Q_1 + Q_2$$.

So yes, if you view $$Q_1$$ as a $$4 \times 4$$ matrix then you will have $$Q_1 = QP_S$$ (and someone please correct me if I'm wrong, but I don't see why $$Q_1 = P_SQ$$ also wouldn't work).