What is the intuitive meaning of the basis of a vector space and the span?

Question

The formal definition of basis is:

A basis of a vector space $V$ is defined as a subset $v_1, v_2, . . . , v_n$ of vectors in that are linearly independent and span vector space $V$.

The definition of spanning is:

A set of vectors spans a space if their linear combinations fill the space.

But what is the intuitive meaning of this, and the idea of a vector span? All I know how to do is the process of solving by putting a matrix into reduced row-echelon form.

Separately, I"m not sure if I should put this in a new question, but could someone relate this to an intuitive explanation for the row space and column space? So a column space is all the linear combinations of each column of matrix $A$. So what? What does this imply? And a row space is, is it a linear combination of all the rows of $A$, because the book just says its the column space of $A^T$, which I hope means the same thing. So, sure, that's what the definitions of the row space and column space are, but how do all these concepts relate? I'm getting especially confused getting to the fundamental theorems of linear algebra part where we talk about row space, column space, and nullspaces all together.

Answer 1

Take, for example $V = \mathbb R ^2$, the $x$-$y$ plane. Write the vectors as coordinates, like $(3,4)$.

Such a coordinate could be written as a sum of its $x$ component and $y$ component: $$(3,4) = (3,0) + (0,4)$$ and it could be decomposed even further and written in terms of a "unit" x vector and a "unit" y vector: $$(3,4) = 3\cdot(1,0) + 4\cdot(0,1).$$ The pair $\{(1,0),(0,1)\}$ of vectors span $\mathbb R^2$ because ANY vector can be decomposed this way: $$(a,b) = a(1,0) + b(0,1)$$ or equivalently, the expressions of the form $a(1,0) + b(0,1)$ fill the space $\mathbb R^2$.

It turns out the $(1,0)$ and $(0,1)$ are not the only vectors for which this is true. For example if we take $(1,1)$ and $(0,1)$ we can still write any vector:

$$(3,4) = 3\cdot(1,1) + 1\cdot(0,1)$$ and more generally $$(a,b) = a \cdot (1,1) + (b-a)\cdot(0,1).$$

This fact is intimately linked to the matrix $$\left(\begin{array}{cc} 1 & 1 \\ 0 & 1 \end{array} \right)$$ whose row space and column space are both two dimensional.

I would keep going, but your question is general enough that I could write down an entire linear algebra course. Hopefully this gets you started.

Answer 2

There is a natural tension between the ideas of linear independence and span. Given a particular $n$ dimensional space, we would like to represent any vector $\mathbf x$ in that space as a linear combination of a set of vectors $V = \{\mathbf v_1,\mathbf v_2, \ldots,\mathbf v_m\}$. So the question is, what type of requirements does this set of vectors need to satisfy for this to be possible? By definition, the set of vectors should span $\mathbb R^n$ meaning that the entire space is reachable with some linear combination of the vectors, so clearly $m\geq n$. Unfortunately the does not yield a definitive test that we can construct $\mathbf x$ from $V$. We can conceive of situations where $V$ contains more than $n$ vectors yet they do not span $\mathbb R^n$ (you should understand that $m\geq n$ is a necessary condition but not a sufficient condition). So if we consider the case where $m > n$ and they span $\mathbb R^n$, then we have too many vectors. How then do we tell which ones are unnecessary? The answer is linear independence. Any vector in $v$ which is a linear combination of the others is unnecessary and can be removed from the set. If we systematically remove all of the linearly dependent vectors from the set $V$, we then have a linearly independent set of vectors which span $\mathbb R^n$. That is a basis. A basis is both linearly independent (it doesn't have too many vectors) and it spans the space (it has enough vectors). Thus the basis strikes a balance between span and linear independence.

Regarding column and row space, you should understand that a multiplication of a matrix times a vector can be interpreted in two different ways. Consider the following $$\mathbf{y=Ax}$$ where $\mathbf{A}$ is $m\times n$ and $\mathbf{x}$ is $n\times 1$. You may interpret this multiplication as

1. The linear combination of the columns of $\mathbf{A}$, or
2. The set of dot products of the rows of $\mathbf{A}$ with the vector $\mathbf{x}$.

Both viewpoints are valid. In view (1), if we think about the columns of $\mathbf{A}$ (column space) forming a basis, we can think about multiplication as a composition of the basis vectors to form a new vector $\mathbf y$. In view (2), if we think about the rows of $\mathbf{A}$ (row space) forming a basis, we can think about the multiplication $\mathbf{Ax}$ as the representation of $\mathbf x$ in that basis (i.e. we are projecting the vector $\mathbf x$ onto each row vector).

Now consider the case where the rows of $\mathbf A$ do not span $\mathbb R^n$. Then there are some values of $\mathbf x$ which are orthogonal to the entire row space. This is called the null space. The fundamental theorem of linear algebra says that the dimension of the row space plus the dimension of the null space is $n$. Can you see why that is? It is because $\mathbb R^n$ is spanned by the row space and the null space, however every vector in the row space is orthogonal to every vector in the null space.

Answer 3

Here is an explanation of spanning in terms of linear equation solving: A system of $n$ linear equations with $m$ variables may be written in the form $$a_1 x_1 + ... + a_m x_m = b$$ where $a_1,...,a_m$ and $b$ are columns of length $n$ (When the equation is given in matrix form $Ax=b$, then $a_1,...,a_m$ are just the columns of the matrix $A$). And taking a second look at this form, $b$ is written as a linear combination of the $a_j$.

This means that $m$ vectors $a_1,...,a_m$ of $\mathbb{R}^n$ span $\mathbb{R}^n$ if and only if the system of equations $Ax=b$ (where $A$ is the matrix formed by $a_1,...,a_m$ as columns) has a solution for all right-hand sides $b\in \mathbb{R}^n$.

Answer 4

To know everything about a vectorspace, just meet Mr. basis!!

Suppose $V$ is a vectorspace of uncountable number of elements over the field $F$ and suppose $V$ is of finite dimensional space. Then any element $\bf{v}$ of $V$ can be represented as linear combination of all elements of $V$.

${\displaystyle \textbf{v}=\sum_{\textbf{u}\in V}\alpha_u \textbf{u}}$, where $\alpha_u\in F$.

This above representation is not unique! (mathematicians don't like) and interestingly, the element $\bf{v}$ is represented by uncountable number of elements of $V$.(no one like such a huge number!)

1. What is the minimum number of elements from $V$ required to represent all the elements of $V$?
2. Whether the representation of $\textbf{v}$ is unique or not?

For example, consider $V=\mathbb{R}^2$ over the field $F=\mathbb{R}$ and $B=\{\bf{w}_1,\bf{w}_2\}$, where $\bf{w}_1=(1,0),\bf{w}_2=(0,1)$. Any element $\displaystyle {\textbf{v}=(x,y)= x \textbf{w}_1+y\textbf{w}_2+\sum_{\textbf{u}\in V-\{w_1,w_2\}}\alpha_u \textbf{u}}$, where $\alpha_u=0 \in F=\mathbb{R}$, for all $\bf{u}$.

-- To remember all details of the uncountable set $\mathbb{R}^2$, it is simple to remember a basis $B$!!

@Answer to Qn 1 and Qn 2: Linear independency of the basis will take care of it.

Answer 5

---

My answer is in the context of $\mathbb{R}^n$, and I will work directly with the usual definitions of basis, span, and linear independence. I believe doing it this way is helpful because this is the perspective of many students taking linear algebra (especially those in a first course): working in Euclidean space, and wanting to understand how the intuition relates directly to the abstract definitions.

---

What is a basis of $\mathbb{R}^n$? It is a set of linearly independent vectors that spans $\mathbb{R}^n$. But then (1) what does being linearly independent mean, and (2) what does it mean to span $\mathbb{R}^n$?

To answer (1), let’s consider a set of vectors $\vec{v_1}, \vec{v_2}, …, \vec{v_k}$ in $\mathbb{R}^n$. To say that $\vec{v_1}, \vec{v_2}, …, \vec{v_k}$ are linearly independent means that the only linear combination of these vectors that give the zero vector is the trivial solution. In other words, if you have $a_1\vec{v_1} + v_2\vec{v_2} + … + a_k\vec{v_k} = \vec{0}$ for some real numbers $a_1, a_2, …, a_k$, then we must have $a_1 = a_2 = … = a_k = 0$. So what does this tell us? Well, in particular it tells us that each of the vectors $\vec{v_1}, \vec{v_2}, …, \vec{v_k}$ cannot be written as a linear combination of the others. For example, $\vec{v_1}$ cannot be written as $b_2\vec{v_2} + ... + b_k\vec{v_k}$ no matter which real numbers $b_2, ..., b_k$ you choose. Why? Because, if we could, then we will have $\vec{v_1} = b_2\vec{v_2} + ... + b_k\vec{v_k}$, and so subtracting the right-hand side from both sides, we get $\vec{v_1} - b_2\vec{v_2} - ... - b_k\vec{v_k} = \vec{0}$. Note that the coefficient of $\vec{v_1}$ is 1, which is not 0. So we have a non-trivial solution to the equation $a_1\vec{v_1} + v_2\vec{v_2} + … + a_k\vec{v_k} = \vec{0}$, contradicting the assumption of linear independence.

So, intuitively, in $\mathbb{R}^2$, if two vectors $\vec{u}, \vec{v}$ are linearly independent, this means $\vec{u}$ cannot be written as $c\vec{v}$ no matter what real number $c$ you choose (and vice versa, $\vec{v}$ cannot be written as $c\vec{u}$). This means $\vec{u}$ doesn't lie in the line that is formed by $\vec{v}$. In $\mathbb{R}^3$, if three vectors $\vec{u}, \vec{v}, \vec{w}$ are linearly independent, this means any one of these vectors cannot be written as a linear combination of the others. For example, $\vec{u}$ cannot be written as $c\vec{v} + d\vec{w}$, in other words, $\vec{u}$ doesn't lie in the plane spanned by $\vec{v}$ and $\vec{w}$. (You can see my answer here for some geometric intuition as to why two linearly independent vectors in $\mathbb{R}^3$ span a plane.) So if you have three vectors lying in the same plane, you know they cannot be linearly independent.

To answer (2): if the span of $\vec{v_1}, \vec{v_2} ..., \vec{v_k}$ equals $\mathbb{R}^n$, what does this mean? It means that the set of all linear combinations $a_1\vec{v_1} + a_2\vec{v_1} + ... + a_k\vec{v_k}$ equals $\mathbb{R}^n$. You can think of spanning $\mathbb{R}^n$ as sort of like a game. If I give you any vector $\vec{v}$ in $\mathbb{R}^n$, can you give me scalars $a_1, a_2, ..., a_k$ so that $\vec{v} = a_1\vec{v_1} + a_2\vec{v_2} + ... + a_k\vec{v_k}$? In other words, if you give me any vector $\vec{v}$ in $\mathbb{R}^n$, can you write it as a linear combination of $\vec{v_1}, \vec{v_2}, ..., \vec{v_k}$? Since $\vec{v_1}, \vec{v_2}, ..., \vec{v_k}$ spans $\mathbb{R}^n$, the answer is yes.

So, what then does it mean if $\vec{v_1}, \vec{v_2}, ..., \vec{v_k}$ is a basis for $\mathbb{R}^n$? It means none of these vectors can be written in terms of the others (i.e. linear independence), and it means that by using linear combinations of these vectors, I can reach any vector in $\mathbb{R}^n$ (i.e. spans $\mathbb{R}^n$).

Linear independence sort of tells you that you don't have any ''extra/unnecessary'' vectors. To be linearly dependent (i.e. not linearly independent) means some vector is in the span of the other vectors, so it is ''extra/unnecessary'' in the sense that we don't need it. Whenever we encounter this vector, we can just replace it as a linear combination of the other vectors. So really all we need are those other vectors.

So, a basis of $\mathbb{R}^n$ is a set of vectors that is able to reach every vector in $\mathbb{R}^n$, but at the same time we don't have any ''extra/unnecessary'' vectors. For example, if we have three vectors in $\mathbb{R}^2$ no two of which lie in the same line, then any two of these vectors will span a plane, i.e. all of $\mathbb{R}^2$. So the third vector is extra/unnecessary. In other words, these three vectors would span $\mathbb{R}^2$, but it is not linearly independent. So they do not form a basis for $\mathbb{R}^2$. If we have two linearly independent vectors in $\mathbb{R}^3$, then they span a plane, which is not all of $\mathbb{R}^3$. So they would be linearly independent, but not span $\mathbb{R}^3$. So they do not form a basis for $\mathbb{R}^3$. A basis is when both these conditions are met. For example, two linearly independent vectors in $\mathbb{R}^2$ form a basis for $\mathbb{R}^2$, and three linearly independent vectors in $\mathbb{R}^3$ form a basis for $\mathbb{R}^3$. More generally, $n$ linearly independent vectors in $\mathbb{R}^n$ form a basis for $\mathbb{R}^n$.