Why a subspace of a vector space is useful

I'm in a linear algebra class and am having a hard time wrapping my head around what subspaces of a vector space are useful for (among many other things!). My understanding of a vector space is that, simplistically, it defines a coordinate plane that you can plot points on and figure out some useful things about the relationship between vectors/points.

I think what I'm curious about is more application of some of these ideas. Such as, is a subspace useful for a reason other than you don't have to look at the entire space something exists in (I guess one way I've been thinking about it is if you want to make a map of a city, you don't necessarily need to make a map of the state it's in) or am I even wrong about that much? Also, even though I feel like I should know this at this point, is if the subspace is linearly independent, is it still a subspace? If it is, what exactly does that describe and/or why is that still useful? If it's not, is it still useful for something?

I think the most difficult part of this for me is I'm having a hard time being able to visualize what exactly we're talking about and I have a hard time thinking that abstractly. I know one or two examples of this might be too specific and doesn't generalize the concept enough, but I think if I have some example to relate back to when applying the idea to new things it might be helpful.

-Frank

-
Remember that there are some sets of vectors that are in $R^3$ but does not span the entire space, so we need to define what is a subspace. –  Lucas Zanella Jun 2 '14 at 2:53
Hello. Sorry, I'm still trying to relate this to something I can visualize. Could, say, the velocity of a ball being thrown be considered a vector in R2 on earth, which we could say is R3. Since there is gravity on earth (and earth is round), the ball couldn't possibly span R3 in any direction, and its distance could then be considered a subspace in R3? –  Frank A. Jun 2 '14 at 4:05
@Frank: hardly anything you just said makes sense. The ball is not a vector, so it can't span anything. The distance (length) of a vector is a real number, not an element of $\mathbb{R}^3$, so "its distance could then be considered a subspace in R3" is entirely meaningless. You need to pay closer attention to definitions. If you want something to visualize: the subspaces of $\mathbb{R}^2$ are either (a) the origin, (b) lines through the origin, or (c) the entire plane. The subspaces of $\mathbb{R}^3$ are either the origin, lines through the origin, planes through the origin, or all of 3-space. –  symplectomorphic Jun 2 '14 at 4:36
Right, I know distance is a scalar. I was thinking the velocity itself. Can the velocity be looked at as some vector in R3 that doesn't span the length of R3? Or is there any relationship between the two at all? I'm just trying to relate the concept to something I can compare it to. –  Frank A. Jun 2 '14 at 4:45
@FrankA. I think it might be best if you just abandon all attempts at physical analogies for a while. Vector spaces are purely abstract objects. They can represent physical space, but the sorts of analogies you're making don't seem accurate at all, and it might be best to just focus on the pure abstraction for a little while until you understand it better. –  Jack M Jun 2 '14 at 13:09

I'd say the the root of the fact that subspaces are important have to do a lot with linear transformations:

It is not hard to show that the nullspace (or kernel) and the image of a linear transformation are vector spaces (i.e. subspaces of the domain and the codomain, respectively).

Since these two things in large part form the basis (no pun intended) of most of linear algebra to some degree, being able to know that you needn't consider anything wild or different when a linear transformation is not injective or surjective is a useful thing.

-
Hello, thank you for your response! You mention kernel and image (two other topics I also have quite a bit of difficulty understanding). Do these two things make up the two vectors in the space in which we can use as a reference for plotting... say in standard xy 2 space, (1,0),(0,1), do the kernel and image make up these vectors for the standard basis? This would also be the subspace, as in the only area we really want to evaluate rather than an entire space something may exist in? Sort of like we map stuff out on earth but not in relation to the entire universe? –  Frank A. Jun 2 '14 at 3:54
The kernel and image are terms that refer to things relative to a linear map. Recall that a linear map is a function $f:V\rightarrow W$ between two vector spaces $V$ and $W$ (over the same field, say $F$) that is linear, in the sense that $f(av+v')=af(v)+f(v')$ for every $v,v'\in V$ and $a\in F$. Then the kernel is the subset of $V$ that gets mapped to zero: i.e. $\null f=\{v\in V\mid f(v)=0\}$, and the image is everything that gets mapped to: $\mathrm{im} f=\{w\in W\mid \text{exists$v\in V$such that$f(v)=w$}\}$. –  Hayden Jun 2 '14 at 3:57

Vector subspaces are important because sub-objects are ever where in math:

• Topological Space $X$: Subspace Topology induced on $Y \subset X$
• Ring $R$: Subring or $R$-SubModule (Ring Ideal)
• Group $G$: Subgroup or Submonid or Subsemigroup
• Monoid $M$: Submonoid or Subsemigroup
• Module $V$: Submodule
(an $\Bbb{R}$-vector space is a special case of module, in which the scalar ring is the field $\Bbb{R}$ (all fields are rings by def.))

The sub-objects usually consist of the subsets that are closed under all the ops of the structure (ring has $2$). Topological subspaces though are simply subsets and a topology can be induced on them by intersecting the subset $Y$ with every open set from the mother space $X$.

You then have topological groups, topological rings, topological fields, topological vector spaces, and so on. These are structures with both the topology and the algebraic structure present, but they're compatible in the sense that the operations defining the algebraic structure are also continuous maps with respect to the topology.

One neat thing about sub-objects in algebra is you can usually take a quotient of the structure $A$ with a sub-object $B$ to form $A/B$ which has important properties that relate back to $A$ and to other structures.

Let $W \leqslant V$ be a subspace of an $\Bbb{R}$-vector space $V$. Then since $W \leqslant V$ forms an additive abelian subgroup, take the quotient group $V/W = \{ v + W : v \in V\}$ under the obvious addition and turn it into a $\Bbb{R}$-vector space by introducing $r \cdot (v + W) := rv + W$. Show that it's also an $\Bbb{R}$-vector space. That was one possible construction for a quotient a vector space with it's subspace.

It has the effect of "zeroing-out" the subspace $W$ as $v + W = W = 0 + W = 0_{V/W} \iff v \in W$.

If you have a linear transformation $T : V \to U$ of $\Bbb{R}$-vector spaces then $V/\ker{T} \approx {\rm im} T$ is a vector space isomorphism, and existence of some isomorphism is an omnipresent idea in all mathematics.

If you have two vector subspaces $W, U \leqslant V$, and you additionally have $W \cap U = \{0\}$ and $W + U = V$ then $W+U$ is written $W\oplus U$ and is called a direct product decomposition of $V$. It has the property that all $v \in V$ can be written in one and only one way as $v = w + u, \ \ w \in W, \ u \in U$, thus $\Bbb{R}^3 \approx \Bbb{R}e_1 \oplus \Bbb{R} e_2 \oplus \Bbb{R} e_3$, where $e_i = \{$ some zeroed vector except for $i$th coordinate $\}$.

These are just basic properties though. The true answer goes deep!

-
This answer already gives lots of good details. The short version would be: if you understand the structure of the subspaces, you understand the vectorspace itself. It's just the components it's build up from. –  Zane Jun 2 '14 at 16:26

An example, among many, of the usefulness of the concept of subspaces is that it is itself a vectorspace. Hence once a vectorspace has been built, one can construct many more examples by considering its vectorspace.

Also, it gives us an easy way to check that a space is a vectorspace. Instead of having to check all the axioms, we may check that some space is a subspace of a vectorspace and then conclude that it is also a vectorspace.

-

It can help to think of these concepts geometrically.

In the context of our 3d world, subspaces might be thought of as lines or planes (through the origin).

Why do we care about subspaces? Again, a geometric picture of linear transformations (which is what we use matrices to model) helps with these ideas. A linear transformation (matrix) might leave certain lines invariant: they simply map the line to the same line, within a scaling factor. Any vector on such a line is an eigenvector, and the scale factor by which the line is magnified or shrunk is the eigenvalue.

A linear transformation (matrix) might, even when given any vector in 3d, only spit out vectors on a certain plane or line. The set of vectors spat out in this way is the image of the transformation, and you should see that the dimensionality of the image is clear from its geometric dimension: if the image is a plane, then the image has dimension 2, and so on.

For inputs to the transformation (matrix), some lines or planes might be wholly annihilated by the transformation---the transformation (matrix) forces them to zero. These lines or planes form the kernel of the transformation (matrix).

Now, something you might not be taught about are whole planes that are left invariant under a transformation, even though no individual line is kept invariant. These planes might be scaled by some factor, and so they can be thought as "eigenplanes". Rotation maps are an example of transformations that leave whole planes invariant without leaving any individual (real) line in that plane invariant.

-

Just one of many examples: You have data points $(x_i,y_i)$ for $i=1,\ldots,n$ and you want to fit a line $y=\hat ax+\hat b$ by least squares (the "hats" --- the diacritics --- on $\hat a$ and $\hat b$ mean that they are estimates, based on a sample, rather than on the whole population). That means you want to choose $\hat a$ and $\hat b$ so as to minimize the sum of squares of the residuals $\hat\varepsilon=y_i - (\hat a x_i+\hat b) = y_i - \hat y_i$.

Then

• The vector $(\hat y_1,\ldots,\hat y_n)$ of "fitted values" is the orthogonal projection of the vector $(y_1,\ldots,y_n)$ of observed values onto the $2$-dimensional subspace spanned by $(x_1,\ldots,x_n)$ and $(1,\ldots,1)$.
• The vector $(\hat\varepsilon_1,\ldots,\hat\varepsilon_n)$ is the orthogonal projection of the vector $(y_1,\ldots,y_n)$ of observed values onto the $(n-2)$-dimensional subspace orthogonal to the $2$-dimensional space mentioned above. (That is why one says there are "$n-2$ degrees of freedom for error".)
• The fact that those two subspaces are orthogonal to each other, conjoined with some commonplace assumptions about the probability distribution of the errors $(\varepsilon_1,\ldots,\varepsilon_n)$ (not to be confused with the residuals $(\hat\varepsilon_1,\ldots,\hat\varepsilon_n)$ mentioned above) makes it possible to conclude that the random variables $\hat b$ and $\hat \varepsilon_1^2+\cdots+\hat\varepsilon_n^2$ are independent of each other, and that in turn is used in finding a confidence interval for $b$ and prediction intervals for future $y$-values.
-

Subspaces are what you get when you take spans. The span of a single non-zero vector is a one-dimensional subspace. The span of two linearly independent vectors is a two-dimensional subspace. And so on. It is a convenient way to talk about "the set of all linear combinations you can get by starting out with a certain set of vectors." Or, in fewer words: the subspace generated by a set of vectors.

For example, the span of $(1,0)$ is the $x$-axis. The only vectors you can get by taking linear combinations of $(1,0)$ are vectors of the form $(c,0)$, which is precisely the $x$-axis.

The span of $(1,0)$ and $(2,0)$ is also the $x$-axis. That's because the second vector is redundant: it is already in the span of the first vector, so it doesn't get you anything the first vector doesn't. To enlarge the span, we'd need a vector that isn't in the span of $(1,0)$. Take $(1,1)$ for example: the span of $(1,1)$ is the line $y=x$, which is not the $x$-axis. And the span of $(1,0)$ and $(1,1)$ is the whole of $\mathbb{R}^2$.

More generally, the subspaces of $\mathbb{R}^2$ are: the origin, lines through the origin, or the entire plane. The subspaces of $\mathbb{R}^3$ are: the origin, lines through the origin, planes through the origin, or the all of 3-space. In general, the subspaces of $\mathbb{R}^n$ will be: the origin, lines through the origin, planes through the origin, and so on, all the way up to the entire space.

In short: the object that we get when we take a span is a subspace, and conversely every subspace can be written as the span of some set of "generators." (If you force these generators to be linearly independent, they form a "basis" for the subspace.)

The other answers have pointed out specific applications of the concept. But the short answer to your question, for elementary purposes, is that it is convenient language: it helps us look at parts of a larger space that are closed under linear combinations.

It's worth pointing out that your language is confused. You ask, for instance:

"if the subspace is linearly independent, is it still a subspace?"

In fact, NO subspace is linearly independent, because every subspace includes the zero vector. Instead, the interesting question is usually whether a finite set of non-zero vectors is linearly independent. For example, the set containing the two vectors $(1,0)$ and $(2,0)$ is not linearly independent, because the second vector is a multiple of the first. But the set containing the two vectors $(1,0)$ and $(1,1)$ is linearly independent.

-
Yeah, I confuse a lot of the terms because I don't think I truly understand their meaning. I know it's supposed to be general because math tends to describe many different things, but it's tough for me. My major is computer science so a few of the concepts from linear algebra make sense because I can relate it to how certain computer parts work (or a couple programming concepts I studied) but otherwise I've been having a lot of difficulty with this subject. I know some of the concepts of LA relate to physics, but I don't know enough physics to really ask a question relating to it. –  Frank A. Jun 2 '14 at 5:05

One particular application of subspaces that I find illuminating is that they reveal the structure of solutions of linear inhomogeneous equations. Say, you have the following equation: $$L(x)=f(x)\tag{1}$$ where $x\in X$; $X$ being some vector space and $L:X\to X$ is a linear operator. Then the most general solution of $(1)$ has the form $x = l+v$ where $l\in U \subset X$, a linear subspace of $X$ spanned by the fundamental solutions of the corresponding homogeneous equation: $$L(x)=0 \tag{2}$$ and $v\in X \setminus V$. For example, if $L=\cfrac{d^2}{dt^2}+\omega^2$, $U$ would be spanned by $\{e^{-\omega t},e^{\omega t}\}$ Once the general structure is clear, if say, you are lucky to know the fundamental system of $(2)$ then you only have to find, perhaps, guess, just one more solution not spanned by the fundamental system to construct the general solution.

This theory is most frequently illustrated on the examples of linear ordinary differential equations, however, I like this example from the domain of the recurrence equations. The problem of expressing function $f(x)=x^2$ in terms of cubic splines on the entire real axis leads to the following equation: $$\lambda_{j}+4\lambda_{j+1}+\lambda_{j+2}=24(j+3)^2$$

Here $L=I+4E+E^2$ where $I$ is the identity and $E$ is the shift operator. The general solution of the corresponding homogeneous relation $$\lambda_{j}+4\lambda_{j+1}+\lambda_{j+2}=0$$ is given by the following expression: $$\lambda^h_j=\alpha\left(-2-\sqrt{3}\right)^{j}+\beta\left(-2+\sqrt{3}\right)^{j}$$ Which is a linear subspace spanned by $\{\left(-2-\sqrt{3}\right)^{j},\left(-2+\sqrt{3}\right)^{j}\}$ It can be found, for example, using generating functions or via the characteristic equation using ansatz $\lambda_j=r^j$. Not surprisingly it depends on 2 arbitrary constants, as it takes 2 initial terms, $\lambda_0$ and $\lambda_{-1}$ to define the sequence from the three-term recurrence, so the subspace is 2-dimensional.

Applying the general ideas from the linear systems we deduce that in order to obtain the general solution of the inhomogeneous recurrence we have to add a particular solution to the expression above.

Since the RHS is the quadratic polynomial it makes sense to look for the particular solution in the form:

$$\lambda^p_j=aj^2+bj+c$$

Substituting this into the original recurrence and gathering together the powers of $j$ and equating them on both sides we obtain the general solution:

$$\lambda_j=\alpha(-2-\sqrt{3})^{j}+\beta(-2+\sqrt{3})^{j}+4j^2+16j+\frac{44}{3}$$

Using initial data $\lambda_0=\frac{44}{3}$ and $\lambda_{-1}=\frac{8}{3}$ to determine the unknown constants we finally get $$\lambda_j=4j^2+16j+\frac{44}{3}$$ which is the solution of the original problem.

-