What is significance of the row space?

Question

I became to study linear algebra a bit on my own. It is already more than twenty years since my graduation. I ended up to my question due to practical problem that I have already solved. Having said that, my practical problem lead me to trying understand a concept of a column space, row space, null space and left null space.

I have developed a intuition or concept of a column space. In my thinking matrix columns are vectors in given space. Further on, in my thinking a column space is a space spanned by matrix columns. All that makes sense to me. I just rely on interpretation of a matrix as collection of vectors. Also, somehow I have become to understand a null space as a special space that will lead to matrix equation of Ax=0. In my thinking, importance of a null space is that there can't be inverse for a such transformation.

Now, my difficulties arise in understanding a row space. It is useless to repeat definition of a row space as Col(A'). The definition doesn't really open this for me. I'd like to understand significance of a row space. In my thinking, columns of a matrix are vectors. This understanding doesn't really fit to a row or a row space. Rows are not vectors, hence a row space is not similar space as a vector space spanned by vectors. Yes, technically it is so or may be, as I have understood. But I am still missing intuition or understanding of concept of a row space. What is significance of a row space, how does it manifest itself?

And very similarly, what is importance of a left null space, how does it manifest itself?

You see, I am not a mathematician nor student of math. I am an engineer. And due to my personal way of learning, I am always trying to visualize or develop some kind of intuition before diving into the details and rigorous definitions.

Thanks in advance

• Poincaré look-alike -

Answer 1

We can think of vectors as either rows or as columns, and can view matrix vector multiplication as acting with either: $x \mapsto Ax$ when $x$ is a column, $v \mapsto vA$ when $v$ is a row. So the column space and row space of a matrix have equal importance, they are essentially interchangeable depending on how we are working with the matrix and vectors.

In fact, you can find entire books which define vectors as rows and not columns, all of the definitions are adjusted slightly to account for this change and everything works as expected.

Sometimes we want a matrix to act on both rows and columns: for example take a bilinear form $B(u,v) = u^{T}Av$. Now consider the following question: is there a vector $u_{0} \neq 0$ such that $B(u_{0},v) = 0$ for all $v \in V$?

We will have $B(u_{0},v) = u_{0}^{T}Av = 0$ for all $v \in V$ precisely when $u_{0}^{T}A = 0$, in other words, when $u_{0}$ is in the left null space of $A$.

Note if you are interested in an advanced perspective on this: the formal connection between these two points of view (row vectors/column vectors) is provided by the concept of the dual space of a vector space; it states (abbreviated form) that if you are working with e.g. column vectors, then you can think of a one-row matrix as a linear function into the base field, and the collection of such functions also form a vector space (that in the finite-dimensional case is isomorphic to the original vector space).

Answer 2

I personally view it from the following perspective: if you have a matrix, you can associate a set of linear equations with it (one row gives you one equation $\sum_{j}a_{ij}x_j=0$). And if you have a bunch of relations, you may want to ask what other linear relations $\sum_j c_jx_j=0$ follow from it? (that is, I write you a bunch of equalities like $x+y+z+t=0, x-y+z=0, y-z+t=0$, etc. but what you really care about in all that jungle is whether you must necessarily have $x=y$). The answer is that the new linear equation follows from your system if and only if the row $c$ defining it is in the row space of your matrix $A$, i.e., there are no non-trivial linear consequences of a system linear equations. In the latter form, the statement can be generalized to linear inequalities and, once you understand what it means there, it tells you pretty much all you need to know about the duality in linear programming though it is not, strictly speaking, about the row space anymore.

That is not the only possible interpretation of it, but I find it quite useful to know and always tell it to the students when teaching a course in linear algebra. See if it clarifies things for you a bit or not :-)

Answer 3

Let's consider an $m \times n$ matrix $A$ of real numbers.

Although the row space is in general different than the column space, we have: \begin{align} \text{A matrix $A \in \mathbb{R}^{m \times n}$ takes its row space onto its column space.} \tag{1} \end{align} Moreover, this is a one-to-one correspondence, so we can invert $A$ on the column space. The column space is an "isomorphic copy" of the row space.

Geometrically, if $A$ is viewed as a linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$, then the subspace spanned by the row space (e.g. a plane) is mapped to a subspace of the same dimension (e.g. another plane).

So this tells us that row rank equals column rank. In any matrix, the number of linearly independent rows is the same as the number of linearly independent columns.

Since you also ask about the left null space $N(A^T)$, here is one way it might be used:

$$\text{Let $A \in \mathbb{R}^{m \times n}$. $Ax = b$ is solvable iff $b$ is orthogonal to $N(A^T)$.} \tag{2} $$

This can be useful sometimes. For example, if we already know a basis of the left null space, then to check whether $b$ is a solution, it suffices to check whether its dot product with each basis vector of the left null space is zero. (An example involving Kirchoff's Voltage Law is given in section 3.1 of Gilbert Strang's Linear Algebra and its Applications.)

---

Proof of (1): The row space $R(A)$ is the orthogonal complement of the null space $N(A)$, so $\mathbb{R}^n = R(A) \oplus N(A)$; that is, every vector $x$ in $\mathbb{R}^n$ can be written uniquely as a sum $x = x_r + x_n$ where $x_r \in R(A)$ and $x_n \in N(A)$.

Given $y$ in the column space $C(A)$, say $y = Ax$ where $x = x_r + x_n$, we have $y = Ax = A(x_r + x_n) = Ax_r + Ax_n = Ax_r$, so every $y \in C(A)$ has a vector $x_r \in R(A)$ mapping to it.

This $x$ is also unique, because if $Ax' = y$ for another $x' = x_r' + x_n'$, then $0 = y - y = Ax - Ax' = Ax_r - Ax_r' = A(x_r - x_r')$. So $x_r - x_r'$ is in the null space of $A$. But $x_r - x_r'$ is also in the row space, and the row space and null space are orthogonal, so $x_r - x_r'$ is orthogonal to itself, hence it must be the zero vector. So $x_r = x_r'$. $\square$

Proof of (2): Note that "$Ax = b$ is solvable" just means that $b$ is in the column space $C(A)$, and "$b$ is orthogonal to $N(A^T)$" just means that $b \in N(A^T)^\perp$. But $C(A) = N(A^T)^\perp$, since the column space is the orthogonal complement of the left null space. $\square$