0
$\begingroup$

I became to study linear algebra a bit on my own. It is already more than twenty years since my graduation. I ended up to my question due to practical problem that I have already solved. Having said that, my practical problem lead me to trying understand a concept of a column space, row space, null space and left null space.

I have developed a intuition or concept of a column space. In my thinking matrix columns are vectors in given space. Further on, in my thinking a column space is a space spanned by matrix columns. All that makes sense to me. I just rely on interpretation of a matrix as collection of vectors. Also, somehow I have become to understand a null space as a special space that will lead to matrix equation of Ax=0. In my thinking, importance of a null space is that there can't be inverse for a such transformation.

Now, my difficulties arise in understanding a row space. It is useless to repeat definition of a row space as Col(A'). The definition doesn't really open this for me. I'd like to understand significance of a row space. In my thinking, columns of a matrix are vectors. This understanding doesn't really fit to a row or a row space. Rows are not vectors, hence a row space is not similar space as a vector space spanned by vectors. Yes, technically it is so or may be, as I have understood. But I am still missing intuition or understanding of concept of a row space. What is significance of a row space, how does it manifest itself?

And very similarly, what is importance of a left null space, how does it manifest itself?

You see, I am not a mathematician nor student of math. I am an engineer. And due to my personal way of learning, I am always trying to visualize or develop some kind of intuition before diving into the details and rigorous definitions.

Thanks in advance

  • Poincaré look-alike -
$\endgroup$
4
  • $\begingroup$ Dear Sir/Madame, I know already the row space is orthogonal complement of the left null space. I can read that form any given text book. What is difficult to find from a text book is explanation. What I am after is intuition or interpretation. Like I can interpret or understand a vector, I'd like to understand significance of a row and row space. $\endgroup$ – Poincaré look-alike Jan 30 at 12:56
  • $\begingroup$ For a real matrix: $M x= 0$ iff each row $v_j$ of $M$ is such that $v_j \cdot x = 0$ (where $\cdot$ is the scalar product between a row and a column vector) iff $x$ is orthogonal to every $v_j$, thus the orthogonal complement of the row space is the null space, and hence the row space is the orthogonal complement of the null space. $\endgroup$ – reuns Jan 30 at 13:03
  • $\begingroup$ Each row in a matrix corresponds to some linear equation (in as many variables as is the number of columns). So that's the interpretation of a row for me. Thats why it is in connection with a null space, because if a vector is a solution of (homological) system of linear equation it is (as I suppose you know) orthogonal to each row (here the scalar product resembles assigning values to the indeterminates in linear equation) $\endgroup$ – dmk Jan 30 at 13:09
  • $\begingroup$ One small note: you claim rows are not vectors I would like to argue that this is incorrect. Having vectors as column is just our habit, it's not written in stone. 1 In more "computer-science" point of view: the vector is just $n$-tuple of values from a given field. 2 In more abstract point of view, vector is anything that can be multiplied by a scalar or added to other vector (and has some good behaviour like associativity and so on...). And rows of matric create a vector space - the row space. So there is no reason to view row as "not a vector". $\endgroup$ – dmk Jan 30 at 13:29
1
$\begingroup$

We can think of vectors as either rows or as columns, and can view matrix vector multiplication as acting with either: $x \mapsto Ax$ when $x$ is a column, $v \mapsto vA$ when $v$ is a row. So the column space and row space of a matrix have equal importance, they are essentially interchangeable depending on how we are working with the matrix and vectors.

In fact, you can find entire books which define vectors as rows and not columns, all of the definitions are adjusted slightly to account for this change and everything works as expected.

Sometimes we want a matrix to act on both rows and columns: for example take a bilinear form $B(u,v) = u^{T}Av$. Now consider the following question: is there a vector $u_{0} \neq 0$ such that $B(u_{0},v) = 0$ for all $v \in V$?

We will have $B(u_{0},v) = u_{0}^{T}Av = 0$ for all $v \in V$ precisely when $u_{0}^{T}A = 0$, in other words, when $u_{0}$ is in the left null space of $A$.

Note if you are interested in an advanced perspective on this: the formal connection between these two points of view (row vectors/column vectors) is provided by the concept of the dual space of a vector space; it states (abbreviated form) that if you are working with e.g. column vectors, then you can think of a one-row matrix as a linear function into the base field, and the collection of such functions also form a vector space (that in the finite-dimensional case is isomorphic to the original vector space).

$\endgroup$
2
  • $\begingroup$ Sir, thank you. Your answer really makes sense to me. Also, it answer to another question. In some data science application I have noticed data organized in form of row vectors. $\endgroup$ – Poincaré look-alike Jan 31 at 19:37
  • $\begingroup$ @Poincarélook-alike Absolutely, this is because data is more easy to reference and work with as a row. A lot of computational computer software uses rows for vectors. $\endgroup$ – Morgan Rodgers Jan 31 at 20:17
1
$\begingroup$

I personally view it from the following perspective: if you have a matrix, you can associate a set of linear equations with it (one row gives you one equation $\sum_{j}a_{ij}x_j=0$). And if you have a bunch of relations, you may want to ask what other linear relations $\sum_j c_jx_j=0$ follow from it? (that is, I write you a bunch of equalities like $x+y+z+t=0, x-y+z=0, y-z+t=0$, etc. but what you really care about in all that jungle is whether you must necessarily have $x=y$). The answer is that the new linear equation follows from your system if and only if the row $c$ defining it is in the row space of your matrix $A$, i.e., there are no non-trivial linear consequences of a system linear equations. In the latter form, the statement can be generalized to linear inequalities and, once you understand what it means there, it tells you pretty much all you need to know about the duality in linear programming though it is not, strictly speaking, about the row space anymore.

That is not the only possible interpretation of it, but I find it quite useful to know and always tell it to the students when teaching a course in linear algebra. See if it clarifies things for you a bit or not :-)

$\endgroup$
0
$\begingroup$

Let's consider an $m \times n$ matrix $A$ of real numbers.

Although the row space is in general different than the column space, we have: \begin{align} \text{A matrix $A \in \mathbb{R}^{m \times n}$ takes its row space onto its column space.} \tag{1} \end{align} Moreover, this is a one-to-one correspondence, so we can invert $A$ on the column space. The column space is an "isomorphic copy" of the row space.

Geometrically, if $A$ is viewed as a linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$, then the subspace spanned by the row space (e.g. a plane) is mapped to a subspace of the same dimension (e.g. another plane).

So this tells us that row rank equals column rank. In any matrix, the number of linearly independent rows is the same as the number of linearly independent columns.

Since you also ask about the left null space $N(A^T)$, here is one way it might be used:

$$\text{Let $A \in \mathbb{R}^{m \times n}$. $Ax = b$ is solvable iff $b$ is orthogonal to $N(A^T)$.} \tag{2} $$

This can be useful sometimes. For example, if we already know a basis of the left null space, then to check whether $b$ is a solution, it suffices to check whether its dot product with each basis vector of the left null space is zero. (An example involving Kirchoff's Voltage Law is given in section 3.1 of Gilbert Strang's Linear Algebra and its Applications.)


Proof of (1): The row space $R(A)$ is the orthogonal complement of the null space $N(A)$, so $\mathbb{R}^n = R(A) \oplus N(A)$; that is, every vector $x$ in $\mathbb{R}^n$ can be written uniquely as a sum $x = x_r + x_n$ where $x_r \in R(A)$ and $x_n \in N(A)$.

Given $y$ in the column space $C(A)$, say $y = Ax$ where $x = x_r + x_n$, we have $y = Ax = A(x_r + x_n) = Ax_r + Ax_n = Ax_r$, so every $y \in C(A)$ has a vector $x_r \in R(A)$ mapping to it.

This $x$ is also unique, because if $Ax' = y$ for another $x' = x_r' + x_n'$, then $0 = y - y = Ax - Ax' = Ax_r - Ax_r' = A(x_r - x_r')$. So $x_r - x_r'$ is in the null space of $A$. But $x_r - x_r'$ is also in the row space, and the row space and null space are orthogonal, so $x_r - x_r'$ is orthogonal to itself, hence it must be the zero vector. So $x_r = x_r'$. $\square$

Proof of (2): Note that "$Ax = b$ is solvable" just means that $b$ is in the column space $C(A)$, and "$b$ is orthogonal to $N(A^T)$" just means that $b \in N(A^T)^\perp$. But $C(A) = N(A^T)^\perp$, since the column space is the orthogonal complement of the left null space. $\square$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.