I am having trouble understanding the relatioship between rows and columns of a matrix.

Say, the following homogeneous system has a nontrivial solution. $$ 3x_1 + 5x_2 − 4x_3 = 0 \\ −3x_1 − 2x_2 + 4x_3 = 0 \\ 6x_1 + x_2 − 8x_3 = 0\\ $$ Let A be the coefficient matrix and row reduce $\begin{bmatrix} A & \mathbf 0 \end{bmatrix}$ to row-echelon form:

$\begin{bmatrix}3&5&-4&0\\-3&-2&4&0\\6&1&-8&0\\ \end{bmatrix} \rightarrow \begin{bmatrix}3&5&-4&0\\0&3&0&0\\0&0&0&0\\ \end{bmatrix}$

$\quad a1 \quad a2 \quad \,a3$

Here, we see $x_3$ is a free variable and thus we can say 3rd column,$\,a_3$, is in $\text{span}(a_1, a_2)$

But what does it mean for an echelon form of a matrix to have a row of $0$'s?

Does that mean 3rd row can be generated by 1st & 2nd rows?

just like 3rd column can be generated by 1st & 2nd columns?

And this raises another question for me, why do we mostly focus on columns of a matrix?

because I get the impression that ,for vectors and other concepts, our only concern is

whether the columns span $\mathbb R^n$ or the columns are linearly independent and so on.

I thought linear algebra is all about solving a system of linear equations,

and linear equations are rows of a matrix, thus i think it'd be logical to focus more on rows than columns. But why?

  • $\begingroup$ Thanks and upvote for introducing me to the term "echolon form". $\endgroup$
    – k.stm
    Jul 19 '13 at 15:10
  • $\begingroup$ What do you mean by “$x_3$ is a free variable”? It certainly can be generated like that, just by reversing the row operations you used to get the echelon form. I don't see what you mean by focusing on columns, though. I don't really focus on rows or columns most of the time. In fact, I often can't tell which are which. Linear algebra is about much more than just solving linear equations. $\endgroup$
    – tomasz
    Jul 19 '13 at 15:13
  • $\begingroup$ "Mostly focusing on columns" is a convention in some texts. Others focus on (and define things in terms of) rows. $\endgroup$ Jul 19 '13 at 15:18
  • $\begingroup$ @tomasz i mean the last row being the rows of 0's, what's the relation between 3rd row and the other two? 3rd row is obviously a different equation than the other 2 rows and yet its coefficient became all 0.. $\endgroup$
    – Belphegor
    Jul 19 '13 at 15:19

Having a row of $0$'s in the row-echelon form means that we were able to write the third row of $A$ as a linear combination of the second and first rows. As it so happens for square matrices, this is true precisely when we can write the columns as a linear combination of each other (that is, when the columns are not linearly independent). If you further reduce this to reduced row-echelon form, you get $$\begin{bmatrix} 1 & 0 & -4/3 & 0\\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{bmatrix}$$ Because the third column lacks a pivot, $x_3$ is our free variable, which means that we can write $a_3$ as a linear combination of the other two columns.

There's a very good reason for focusing on the columns of a matrix. This comes out of using $A$ as a linear transformation, where the "column space" gives us the "range" of the function $f(\vec x) = A \vec x$.

  • $\begingroup$ " Because the third row lacks a pivot, $x_3$ is our free variable, which means that we can write $a_3$ as a linear combination of the other two columns." - I fail to see the connection between the row of zeros unfortunately. I believe it's true, but I don't have the intuition for it. $\endgroup$
    – Max
    Sep 11 at 16:40
  • $\begingroup$ @Max I had a typo, see my latest edit $\endgroup$ Sep 11 at 17:27

Excellent question.

In some sense, the equations and variables represent equivalent information. Sometimes it is easier to approach the problem from the point of view of rows-equations-constraints, and sometimes - from the point of view of variables.

This is very deeply related to the notion of the dual problem in linear programming. That is exactly what converts equations/constraints to rows and vice versa, looking at a different problem.

Mathematically, this boils down to either working with the matrix $A$ or with some form of $A^T$, which converts rows into columns and columns into rows. Not surprisingly, the main characteristic properties for $A$ and $A^T$ are the same, like rank, eigenvalues/singular values, determinant, trace, etc.

Two reference links on duality:

  1. Wikipedia article on duality
  2. MIT notes on duality in Linear Programming

Clearly, you matrix is singular, i.e. its determinant is zero. When we study the linear system $Ax=b$, we have two options: either there's an affine space of solutions ($b\in Im A$) or there're no solutions at all.

When you study the echolon form, you eventually arrive to the line of the form $\begin{pmatrix}0&0&0&a\end{pmatrix}$. If $a\ne 0$, then the system is incompatible (zero combination of variables produces non-zero). If $a=0$, then you have an affine subspace of solutions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.