# Why are the four fundamental subspaces fundamental?

The four fundamental subspaces in linear algebra, as discussed by Gilbert Strang [1], are the kernel, image, dual space kernel, and dual space image (nullspace, column space, left nullspace, row space). He calls the relationship between these "the fundamental theorem of linear algebra".

Of course the kernel and image are needed for the first isomorphism theorem, which is fundamental to understanding what a homomorphism is.

But why are the dual space image and kernel important and "fundamental"?

A similar question was asked [2] with one answer stating

In short, these four spaces (really just two spaces, with a left and a right version of the pair) carry all the information about the image and kernel of the linear transformation that A is affecting, whether you are using it on the right or on the left.

but this entirely avoids the issue of why the image and kernel of the adjoint of a transformation are important to understanding the transformation.

Strang's paper [1] seems to suggest that these subspaces may provide intuition for the singular value decomposition, but I haven't studied the singular value decomposition (SVD) yet. I'm also not sure if the SVD is fundamental enough to justify classifying the dual space kernel and image as "fundamental".

edit: Thanks for the replies :)

To the men and women of the future that take the path I have trodden on, I'll leave some links that I found particularly helpful [3, 4, 5].

This is more of an intuitive perspective, based on the vector spaces involved:

The null space of a matrix is a subspace of the same vector space that the row space is a subspace of. In this sense the row space of a matrix determines the null space, as we can define the null space of a matrix $A$ as $\{u:vu=0$ where $v^t$ is in the row space of $A\}$. This complementary relationship certainly is fundamental in characterizing a matrix.

If you are looking for applications where row space is important: suppose we have an $m \times n$ transformation $G$ and an $k \times n$ matrix $F$, and we want to know whether $F$ divides $G$ on the right, that is whether there exists $H$ such that $G=HF$. It turns out that this is true iff the row space of $G$ is a subspace of the row space of $F$.

What about column space...as with the relationship between row space and null space, the complementary space to column space is the null space of $A^*$, and the larger vector space context (which these spaces are subspaces of) in this case is different from that of the vector space containing row space and null space, in the case where a matrix is not square.

A very important application of this space which is complementary to column space, is least squares approximation for an over specified system. For a least squares solution we have $$A^*(Ax_0-y)=0,$$ that is for $x_0$ to be a least squares solution we must have $Ax_0-y$ is in the left kernel! Intuitively, to find the point $Ax_0$ where the column space of $A$ is closest to $y$ we must find a vector in the orthogonal complement of the column space of $A$ that contains $y$ and $Ax_0$ (thereby minimizing the distance).

So different contexts place emphasis on different fundamental subspaces. In most common applications there is probably just more of an emphasis on null space and column space.

Suppose you are trying to solve $Af=g$ where $A$ is a linear operator. As always, there are two big issues: existence and uniqueness. Start with a Hilbert space for the sake of discussion.

Existence: A solution of $Af=g$ exists iff $g$ is in the range of $A$. If $A$ happens to have a closed range, then the issue of whether or not $g$ is in the range is settled by looking at the kernel of $A^*$: $$\mathcal{R}(A)=\mathcal{N}(A^*)^{\perp}$$ So, in order to obtain a solution of $Af=g$, you must impose conditions on $g$: $$(g,k) = 0 \mbox{ for all } k \in \mathcal{N}(A^*).$$ For a lot of differential equations, you find $\mathcal{N}(A^*)$ is finite-dimensional, and so a finite number of conditions on $g$ will lead to a solution.

Uniqueness: If you have one solution of $Af=g$, then you can find all others by adding an arbitrary element of $\mathcal{N}(A)$ to $f$. That is, $$Af_1 =g ,\; Af_2 = g \implies f_1 - f_2 \in \mathcal{N}(A).$$ Conversely, if you have a solution $f$ of $Af=g$, then $f+n$ is also a solution for all $n \in \mathcal{N}(A)$. So, uniqueness may also require adding some conditions.

After thinking about this for a while more, I'd like to write done some thoughts.

I'll take for granted that we want to study homomorphisms between vector spaces as deeply as possible.

Now, as is routine with algebra, we can consider the kernel and image and the first isomorphism theorem.

In the finite dimensional case, I find there to be nothing fundamental about dual spaces in themselves. Rather, I think they should be viewed as a stepping stone to the adjoint matrix.

Now multiplication by a matrix is known to have two interpretations: A change of basis (vector stays the same, coordinates change), and a transformation (vector changes, coordinates stay the same). The left hand side of

$$[Tv, w'] = [v, T'w']$$

corresponds to the first view, while the right hand side corresponds to the second because

$$Tv = \sum_i [Tv, e_i'] e_i$$

and

$$v = \sum_i [v, T'e_i'] T^{-1} e_i.$$

It is clear from the latter equation that the dual basis $e_i'$ change inversely to the bases $e_i$. This means that the coordinates in the matrix representation of $v$ change inversely to the bases $e_i$. Since the change of coordinates are so important, this insight in itself motivates the adjoint and the discussion of dual bases.

However, the main payoff of adjoints are found when we consider inner product spaces. If we introduce an ismorphism between $V$ and $V^\ast$ we get an inner product and hence the formula for a self adjoint matrix

$$<Tv, w> = <v, T^\ast w>$$

which otherwise is unmotivated.

Now that we have an inner product, we can consider the image $\mathrm{Im}\; T^\ast$ of our matrix $T^\ast:W\to V$ to be a subset of $V$ rather than $V^\ast$ and compare it to the kernel $\mathrm{Ker}\; T$ of $T:V \to W$. This gives us a method of finding the orthogonal complement to $\mathrm{Im}\; T^\ast$, which is a subset of $V$ rather than $V^\ast$. Finding orthogonal vectors is useful in optimization methods like least squares.

• Actually, you need inner products to consider adjoints at all. An inner product on a vector space $V$ gives an isomorphism $V \to V^*$, and we normally don't have any canonical choice of such a thing. If we now look at $\mathrm{Hom}(V,W) = V^* \otimes W$, then we see that its dual space is $\mathrm{Hom}(V,W)^* = \mathrm{Hom}(W,V)$. Picking inner products on $V$ and $W$ now gives a canonical isomorphism $\mathrm{Hom}(V,W) \to \mathrm{Hom}(W,V)$, which is exactly the map that sends a linear morphism to its adjoint. When evaluated on vectors, this morphism gives exactly your formula for $T^*$. Jun 13 '16 at 15:35
• @GunnarÞórMagnússon There are multiple definitions of the adjoint. The one adopted by Halmos in Finite Dimensional Vector Spaces always exists and is a map from $W^\ast$ to $V^\ast$ for a map $T:V \to W$. I think this is also the standard terminology used in functional analysis and category theory. The other convention, as adopted in Axler's Linear Algebra Done Right (the latest edition, which has extra content) is to refer to the adjoint in Halmos' sense as a dual map and the adjoint in your sense as the adjoint. In an inner product space, the two are equivalent up to isomorphism.
– LMZ
Jun 14 '16 at 4:09
• The one Halmos is talking about is the transpose or dual map of a linear morphism $T : V \to W$. Like you say, it's a linear map $W^* \to V^*$. The adjoint is a linear map $W \to V$. The dual and the adjoint are elements of vector spaces that are not canonically isomorphic to each other. If we give $V,W$ inner products, then we get an isomorphism between those spaces, but it's nowhere near canonical. These really are different things, and just because they look more or less the same on the matrix level once we pick orthonormal bases is no reason to treat them as the same. Jun 14 '16 at 11:50
• You may think this is unreasonable hair-splitting, but there are cases where this stuff matters. Suppose for example that we're doing algebraic or complex geometry. We're doing complicated manipulations of some long exact sequences that come from some vector bundles. Now, if we had an isomorphism between the two, our life would be easier. And we notice that we have an isomorphism of the vector bundles, so we win. Except we don't, because the isomorphism is noncanonical and doesn't play well enough with our functorial constructions. Jun 14 '16 at 12:04
• I agree with you that the adjoint and dual map, in the sense you use them, are different things. But there are two definitions of the word adjoint. So some texts use the word adjoint synonymously with dual map (e.g. Halmos, Katznelson "A terse introduction to linear algebra") while some texts use the word in your sense, so that the adjoint is defined using an inner product (e.g. Axler, probably most modern texts on linear algebra). Here's a draft of Katznelson's text, if you want to verify this for yourself
– LMZ
Jun 15 '16 at 1:04