# How does a vector b in the column space come from a vector in the row space?

I'm working through Gilbert Strang's Introduction to Linear Algebra book and am really confused by a paragraph from chapter 4.1 titled 'Orthogonality of the Four Subspaces'. The paragraph is as follows:

Every vector goes to the column space! Multiplying by A cannot do anything else. More than that: Every vector $$b$$ in the column space comes from one and only one vector $$x_r$$ in the row space. Proof: If $$Ax_r = Ax'_r$$, the difference $$x_r - x'_r$$ is in the nullspace. It is also in the row space, where $$x_r$$ and $$x'_r$$ came from. This difference must be the zero vector, because the nullspace and row space are perpendicular. Therefore $$x_r = x'_r$$.

Further on in the book an exercise is given, where we have to demonstrate this using the following figure: Two pairs of orthogonal subspaces, with the following matrix: $$A = \begin{bmatrix}1 & 2\\3 & 6\end{bmatrix}$$. The column space of the matrix is: $$(1, 3)$$, and its row space is: $$(1, 2)$$. If I multiply A with the randomly chosen $$x$$ vector: $$(1, 1)$$, I arrive at $$b = (3, 9)$$. However, this $$b$$ seems unable to be recreated using a multiple of the row space vector: $$(1, 2)$$. I'm really confused by this. I also feel like I'm missing the meaning of the proof and am not familiar with the $$'$$ symbol in $$Ax'_r$$. Does it mean the transpose?

Any help would be greatly appreciated!

## 3 Answers

The comma in the highlighted sentence in your question really shouldn’t be there. (I have to wonder if it’s really there in the original.) Having that comma there makes it seem like it’s saying that every element of the column space has a unique preimage, which happens to lie in the row space. What the author is actually trying to say is that for every vector $$b$$ in the column space, there is exactly one element $$x$$ of the row space that gets mapped to it. There might well be other vectors in the domain that also get mapped to $$b$$: in fact, the sum of $$x$$ and any element of the null space also gets mapped to $$b$$, and no other vectors do.

That’s exactly what’s going on in the exercise. The column space of $$A$$ is spanned by $$(1,3)^T$$, its row space is spanned by $$(1,2)^T$$ and its null space by $$(2,-1)^T$$. You can find the unique multiple of $$(1,2)^T$$ that is mapped to $$A(1,1)^T=(3,9)^T$$ by solving $$kA(1,2)^T = k(5,15)^T = (3,9)^T$$ for $$k$$, namely $$k=3/5$$. Now, $$(1,1)^T$$ is obviously not an element of the row space, as you’ve noted, but we have $$\begin{bmatrix}1\\1\end{bmatrix} - \frac35\begin{bmatrix}1\\2\end{bmatrix} = \begin{bmatrix}\frac25\\-\frac15\end{bmatrix} = \frac15\begin{bmatrix}2\\-1\end{bmatrix},$$ so their difference is indeed an element of the null space, as claimed.

• I think the pieces are falling into place. I don't think I understand it fully just yet. But going to return to this later to digest it further. Thanks a lot for your answer! – GFlow Mar 18 '20 at 9:30

This theorem is strange, because its not always true... It only holds when the matrix $$\mathbf{A}$$ has full rank. So probably context is missing here.

Anyway, to your question: The row space that is spanned by your example matrix is NOT $$\text{span}\left(\begin{bmatrix}1\\2\end{bmatrix}\right),$$ it is $$\text{span}\left(\begin{bmatrix}1\\2\end{bmatrix}, \begin{bmatrix}3\\6\end{bmatrix}\right).$$

You have two linear independent rows (this is important for the theorem to work!), so you can span $$\mathbb{R^2}$$.

But the underlying meaning here is: $$\mathbf{A}\begin{bmatrix}x_1\\x_2\end{bmatrix}=\mathbf{A}_1x_1+\mathbf{A}_2x_2$$ That means that regardless of what you put in as $$x$$, you will get a linear combination of the columns of $$\mathbf{A}$$, so you are in the column space of $$\mathbf{A}$$.

The theorem now says that if you columns are linear independent, for each element in the column space there is exactly one $$\begin{bmatrix}x_1\\x_2\end{bmatrix}$$ that will lead to this vector by computing $$\mathbf{A}\begin{bmatrix}x_1\\x_2\end{bmatrix}$$.

If that is not true, then you have a null space of $$\mathbf{A}$$, but the null space will always be orthogonal to the column space of $$\mathbf{A}$$.

I hope that cleared some things up. If not, please ask!

• That definitely cleared things up. The thing I overlooked, I believe, is that I could multiply the original (non-transposed) matrix with any vector from the rowspace to arrive at b. But this only works when A has full rank, as you mentioned. However, I'm confused about my mistake. As far as I understand: rank of column space is the same as the rank of row space. You mention we have two linearly independent rows, which I can indeed see. However, if I eliminate both matrices (A and its transpose), I arrive at one pivot row for both. Indicating only one independent column/row, and rank 1. – GFlow Mar 18 '20 at 8:05
• The rows of $A$ are most certainly not linearly independent: $(3,6)=3(1,2)$. – amd Mar 18 '20 at 8:39
• I dont understand how you would eliminate both matrices, what exactly are you doing? As amd said, the rank here is definitely 2 – Nurator Mar 18 '20 at 8:51
• Isn't @amd saying the matrix is rank 1 by saying the two columns are not independent? By eliminating I mean reducing the matrix to its row echelon form. – GFlow Mar 18 '20 at 8:58
• Oh god sorry, yes of course. They are dependent and you have rank 1. Sorry – Nurator Mar 18 '20 at 9:28

The $$'$$ doesn't denote transpose in this case; it's just an adornment to indicate that $$x_r$$ and $$x_r'$$ are two vectors—the author have equally well called them $$x_r$$ and $$y_r$$.

Note that $$A\begin{bmatrix}1\\2\end{bmatrix} = \begin{bmatrix}5\\15\end{bmatrix}$$, which is a multiple of $$b=\begin{bmatrix}3\\9\end{bmatrix}$$. Since matrix multiplication is linear, we can set $$x_r=\frac35\begin{bmatrix}1\\2\end{bmatrix} = \begin{bmatrix}3/5\\6/5\end{bmatrix}$$, which is in the row space, and for which $$Ax_r=\frac35\begin{bmatrix}5\\15\end{bmatrix} = b$$.

• Thanks for clearing things up. As I mentioned in the comment on Nurator's post, this helped me see that I had to multiply the original matrix by a vector from the rowspace, to arrive at b. Not by simply taking a multiple of the row vector. – GFlow Mar 18 '20 at 8:09