Geometrical interpretation of the solution for finding a consistent linear system

Question

in our lecture we learned how to check if a linear system A $\cdot \vec x = \vec b$ is consistent for every $\vec b$.

Example:

$A=\begin{bmatrix}1&3&4\\-4&2&-6\\-3&-2&-7\end{bmatrix}, b=\begin{bmatrix}b_1\\b_2\\b_3\end{bmatrix}$

After doing a couple of row operations this leads to:

$A=\begin{bmatrix}1&3&4\\0&14&10\\0&0&0\end{bmatrix}, b=\begin{bmatrix}b_1\\4\cdot b_1 + b_2\\b_1-1/2\cdot b_2+b_3\end{bmatrix}$

So for the linear system to be consistent it has to follow the constraint:

$b_1-1/2\cdot b_2+b_3=0\ \ \ \ \ \ \ \ \ (1)$

Everything is clear until here. But then there comes this interpretation which I thought I understood but I don't understand it anymore:

The columns of A span the plane $x_1-1/2\cdot x_2+x_3=0\ \ \ \ \ \ \ \ \ (2)$

Why do the columns of A span equation (1)?

And there is this other example without a soltution:

$\begin{bmatrix}2&-2\\3&3\\4&-4\end{bmatrix} \begin{bmatrix}x_1\\x_2\end{bmatrix}\begin{bmatrix}b_1\\b_2\\b_3\end{bmatrix}$

After a couple of row operations I get:

$\begin{bmatrix}1&-1&1/2b_1\\0&2&1/3b_2-1/2b_1\\0&0&1/4b_3-1/2b_1\end{bmatrix}$

So the constraint is:

$b_3-2b_1=0\ \ \ \ \ \ \ \ \ (3)$

If I would interpret this in the same way it would mean that the vectors of matrix A span the line (no plane anymore) represented by equation (3). Can this actually be right? Looking at the last matrix I have two linear indepent vectors so my first guess would be that they span a plane. Also $x_3$ doesn't exist in this equation so why would $b_3$ influence anything? I would really appreciate your help in understanding this.

Answer 1

Thinking purely geometrically for a second (i.e. in terms of vectors and linear transformations, and forgetting matrices and entries), what does as equation like $A\vec x=\vec b$ mean?

It means we have a linear transformation, and a desired result, and we want to know whether we can find an input to the linear transformation that gives us the desired result as a result.

The transformation has an image. This is, by definition, the collection of all possible results from applying $A$ to different vectors in its domain. If $\vec b$ is in this image, then we can find an $\vec x$, and if $\vec b$ is not in this image, then we can't. This is the basic result to have in mind as we start to transition back into thinking about matrices and entries.

The prerequisite of thinking about matrices and entries is to have selected a basis. Let's say we are in your first example for now, so we only have to pick one (the domain and codomain of the linear transformation $A$ is the same space, so we use the same basis for both), say $\vec i,\vec j,\vec k$. Once we have a basis, we can start analysing the linear transformation in terms of this basis.

Take any vector $v$. We can decompose it as $v_1\vec i+v_2\vec j+v_3\vec k$ (this is usually written using the shorthand $\vec v=[v_1,v_2,v_3]^T$, and we call this a column vector). Then in order to figure out what $A\vec v$ is, we can just apply $A$ to this decomposition to get $$ A\vec v=v_1\cdot A\vec i+v_2\cdot A\vec j+v_3\cdot A\vec k\tag{1} $$ We see from this that $A\vec v$ necessarily must be in the span of $A\vec i, A\vec j, A\vec k$, and the coefficients of $\vec v$ carry over to this linear combination. So these three vectors together are enough to describe exactly what $A$ does to any vector.

Now let's say that $A$ is such that $$ A\vec i=1\vec i-4\vec j-3\vec k\\ A\vec j=3\vec i+2\vec j-2\vec k\\ A\vec k=4\vec i-6\vec j-7\vec k $$ Written as column vectors, they become $$ \begin{bmatrix}1 \\-4 \\-3 \end{bmatrix},\quad \begin{bmatrix}3 \\2 \\-2 \end{bmatrix},\quad \begin{bmatrix} 4\\ -6\\-7 \end{bmatrix}\tag{2} $$ By the paragraph above, these are fundamental in describing numerically how $A$ works in relation to the given basis. So, we put them together in a $3\times3$ table, call it a matrix, and let equation $(1)$ and the vectors in $(2)$ decide how a matrix and a column vector creates a new column vector. We call this the "matrix-vector product".

With this view in mind, it is almost immediate that the result of $A\vec x$, no matter what $\vec x$ is, must lie in the span of the columns of $A$. So in the equation $A\vec x=\vec b$, if $\vec b$ is in this span, then the equation is consistent, while if $\vec b$ isn't in the span, then the equation is inconsistent.

This also goes in reverse: If the equations are consistent, then $\vec b$ must necessarily be in the span. So if you have figured out which $\vec b$'s make the equation consistent, you have found the span of the columns of $A$.

If the domain and codomain of $A$ are different, then you have to find one basis for each of the two spaces, and go through basically the same argument.