How does a vector b in the column space come from a vector in the row space?

Question

I'm working through Gilbert Strang's Introduction to Linear Algebra book and am really confused by a paragraph from chapter 4.1 titled 'Orthogonality of the Four Subspaces'. The paragraph is as follows:

Every vector goes to the column space! Multiplying by A cannot do anything else. More than that: Every vector $b$ in the column space comes from one and only one vector $x_r$ in the row space. Proof: If $Ax_r = Ax'_r$, the difference $x_r - x'_r$ is in the nullspace. It is also in the row space, where $x_r$ and $x'_r$ came from. This difference must be the zero vector, because the nullspace and row space are perpendicular. Therefore $x_r = x'_r$.

Further on in the book an exercise is given, where we have to demonstrate this using the following figure: Two pairs of orthogonal subspaces, with the following matrix: $A = \begin{bmatrix}1 & 2\\3 & 6\end{bmatrix}$. The column space of the matrix is: $(1, 3)$, and its row space is: $(1, 2)$. If I multiply A with the randomly chosen $x$ vector: $(1, 1)$, I arrive at $b = (3, 9)$. However, this $b$ seems unable to be recreated using a multiple of the row space vector: $(1, 2)$. I'm really confused by this. I also feel like I'm missing the meaning of the proof and am not familiar with the $'$ symbol in $Ax'_r$. Does it mean the transpose?

Any help would be greatly appreciated!

Answer 1

The comma in the highlighted sentence in your question really shouldn’t be there. (I have to wonder if it’s really there in the original.) Having that comma there makes it seem like it’s saying that every element of the column space has a unique preimage, which happens to lie in the row space. What the author is actually trying to say is that for every vector $b$ in the column space, there is exactly one element $x$ of the row space that gets mapped to it. There might well be other vectors in the domain that also get mapped to $b$: in fact, the sum of $x$ and any element of the null space also gets mapped to $b$, and no other vectors do.

That’s exactly what’s going on in the exercise. The column space of $A$ is spanned by $(1,3)^T$, its row space is spanned by $(1,2)^T$ and its null space by $(2,-1)^T$. You can find the unique multiple of $(1,2)^T$ that is mapped to $A(1,1)^T=(3,9)^T$ by solving $kA(1,2)^T = k(5,15)^T = (3,9)^T$ for $k$, namely $k=3/5$. Now, $(1,1)^T$ is obviously not an element of the row space, as you’ve noted, but we have $$\begin{bmatrix}1\\1\end{bmatrix} - \frac35\begin{bmatrix}1\\2\end{bmatrix} = \begin{bmatrix}\frac25\\-\frac15\end{bmatrix} = \frac15\begin{bmatrix}2\\-1\end{bmatrix},$$ so their difference is indeed an element of the null space, as claimed.

Answer 2

This theorem is strange, because its not always true... It only holds when the matrix $\mathbf{A}$ has full rank. So probably context is missing here.

Anyway, to your question: The row space that is spanned by your example matrix is NOT $$\text{span}\left(\begin{bmatrix}1\\2\end{bmatrix}\right),$$ it is $$\text{span}\left(\begin{bmatrix}1\\2\end{bmatrix}, \begin{bmatrix}3\\6\end{bmatrix}\right).$$

You have two linear independent rows (this is important for the theorem to work!), so you can span $\mathbb{R^2}$.

But the underlying meaning here is: $$\mathbf{A}\begin{bmatrix}x_1\\x_2\end{bmatrix}=\mathbf{A}_1x_1+\mathbf{A}_2x_2$$ That means that regardless of what you put in as $x$, you will get a linear combination of the columns of $\mathbf{A}$, so you are in the column space of $\mathbf{A}$.

The theorem now says that if you columns are linear independent, for each element in the column space there is exactly one $\begin{bmatrix}x_1\\x_2\end{bmatrix}$ that will lead to this vector by computing $\mathbf{A}\begin{bmatrix}x_1\\x_2\end{bmatrix}$.

If that is not true, then you have a null space of $\mathbf{A}$, but the null space will always be orthogonal to the column space of $\mathbf{A}$.

I hope that cleared some things up. If not, please ask!

Answer 3

The $'$ doesn't denote transpose in this case; it's just an adornment to indicate that $x_r$ and $x_r'$ are two vectors—the author have equally well called them $x_r$ and $y_r$.

Note that $A\begin{bmatrix}1\\2\end{bmatrix} = \begin{bmatrix}5\\15\end{bmatrix}$, which is a multiple of $b=\begin{bmatrix}3\\9\end{bmatrix}$. Since matrix multiplication is linear, we can set $x_r=\frac35\begin{bmatrix}1\\2\end{bmatrix} = \begin{bmatrix}3/5\\6/5\end{bmatrix}$, which is in the row space, and for which $Ax_r=\frac35\begin{bmatrix}5\\15\end{bmatrix} = b$.