95
$\begingroup$

I am looking for an intuitive explanation as to why/how row rank of a matrix = column rank. I've read the proof at http://en.wikipedia.org/wiki/Rank_of_a_linear_transformation and I understand the proof, but I don't "get it". Can someone help me out with this ?

I find it hard to wrap my head around the idea of how the column space and the row space is related at a fundamental level.

$\endgroup$
  • 2
    $\begingroup$ Maybe Jordan's canonical form might be of some help. According to that theory, up to similitude every matrix $M$ can be written as $$M=D+N, $$where $D$ is diagonal and $N$ is nilpotent. Transposing, we get $$M^T=D+N^T, $$ so the matrix and its transpose have the same diagonal part. In particular $M$ and $M^T$ share any information that is carried by the diagonal part, and the (column) rank is one of them. This looks like excessively complicated but I cannot think of any simpler explanation. $\endgroup$ – Giuseppe Negro Mar 17 '13 at 16:55

14 Answers 14

51
$\begingroup$

You can apply elementary row operations and elementary column operations to bring a matrix $A$ to a matrix that is in both row reduced echelon form and column reduced echelon form. In other words, there exist invertible matrices $P$ and $Q$ (which are products of elementary matrices) such that $$PAQ=E:=\begin{pmatrix}I_k\\&0_{(n-k)\times(n-k)}\end{pmatrix}.$$ As $P$ and $Q$ are invertible, the maximum number of linearly independent rows in $A$ is equal to the maximum number of linearly independent rows in $E$. That is, the row rank of $A$ is equal to the row rank of $E$. Similarly for the column ranks. Now it is evident that the row rank and column rank of $E$ are identical (to $k$). Hence the same holds for $A$.

$\endgroup$
  • 25
    $\begingroup$ +1 Nice explanation. This essential point of this argument is that elementary row operations, which by construction don't alter the row rank, also no not alter the column rank (and similarly for column operations). That this is so is because doing an elementary row operation just amounts to expressing all columns in a different basis. $\endgroup$ – Marc van Leeuwen Mar 18 '13 at 5:40
  • $\begingroup$ What is n? Also, isn't it true that you can have things other than zeros and ones in the reduced echelon form? $\endgroup$ – user2820379 Mar 13 '15 at 0:15
  • 1
    $\begingroup$ @user2820379 $n$ is the size of $A$. Also, as emphasized in the answer, $A$ is reduced to a matrix that is in both row echelon form and column echelon form. Therefore, everything off-diagonal would be zeroed out (by a row operation or a column operation) and what remain on the diagonal are ones and zeros. $\endgroup$ – user1551 Mar 13 '15 at 22:40
16
$\begingroup$

One way to view the rank $r$ of a $n\times m$ matrix $A$ with entries in a field $K$, is that it is the smallest number such that one can factor the linear map $f_A:K^m\to K^n$ corresponding to $A$ through an intermediate space of dimension$~r$, in other words, as a composition $K^m\to K^r\to K^n$ (taking $C$ and $B$ as the matrices corresponding to the two steps, this means that one has a decomposition $A=BC$ of $A$ as the product of a $n\times r$ and a $r\times m$ matrix). Now one can always factor $f_A$ though the image $\operatorname{Im}f_A\subseteq K^n$, as $K^m\to\operatorname{Im}f_A\hookrightarrow K^n$, and on the other hand this image can never have a dimension larger than a space through which $f_A$ factors; therefore the rank is equal to $\dim\operatorname{Im}f_A$. But that dimension is equal to the maximal number of independent columns of $A$, its column rank.

On the other hand one can view the rows of $A$ as linear functions on $K^m$ that describe the coordinates of $f_A(x)$ as a function of $x$, and the row rank $s$ is the maximum number of independent such functions; once such an independent set of $s$ independent rows is chosen, the remaining coordinates of $f_A(x)$ can each be described by a fixed linear combination of the chosen coordinates (because their rows are such linear combinations of the chosen rows). But this means that one can factor $f_A$ through $K^s$, with the map $K^s\to K^n$ reconstructing the dependent coordinates. The chosen coordinates are independent, so there is no nontrivial relation between them, and the map $K^m\to K^s$ is therefore surjective. This means that $f_A$ cannot factor through a space of smaller dimension than $s$, so the row rank $s$ is also equal to the rank of $A$.

Instead of that separate argument involving the row rank, you can also interpret the row rank of $A$ as the column rank of the transpose matrix $A^t$. Now one can factor $A=BC$ if and only if one can factor $A^t=C^tB^t$; then the minimal $r$ such that one write $A=BC$ with $B\in M_{n,r}$ and $C\in M_{r,m}$ (the column rank of $A$) obviously equals the minimal $r$ such that one write $A^t=C^tB^t$ with $C^t\in M_{m,r}$ and $B^t\in M_{r,n}$ (the column rank of $A^t$, and row rank of $A$).

$\endgroup$
13
$\begingroup$

Define the rank of a matrix $A$ as the largest size of any square submatrix (minor) with non-null determinant. Then if you see the columns of $A$ as vectors, the rank of $A$ can be thought of as the maximal number of linearly independent such vectors. Finally, note that $det(M)=det(M^T)$, and if $M$ is a minor of $A$, then $M^T$ is a minor of $A^T$.

$\endgroup$
13
$\begingroup$

This post is quite old, so my answer might come a bit late. If you are looking for an intuition (you want to "get it") rather than a demonstration (of which there are several), then here is my 5c.

If you think of a matrix A in the context of solving a system of simultaneous equations, then the row-rank of the matrix is the number of independent equations, and the column-rank of the matrix is the number of independent parameters that you can estimate from the equation. That I think makes it a bit easier to see why they should be equal.

Saad.

$\endgroup$
12
$\begingroup$

Let $V,W$ be two vector spaces such that $\dim(V)=n$

The column rank of a matrix $A=\alpha_{(V,W)}(\varphi)$ is equal to the dimension of the image of the application $\varphi$, so

$$\text{column rank}=\dim(\text{im}(\varphi))$$

On the other side, the row rank has to do with the dimension of the kernel of $\varphi$, namely $$\dim(\ker(\varphi))=n-\text{row rank}$$ Now, we know from rank-nullity theorem that

$$\dim(\ker(\varphi))+\dim(\text{im}(\varphi))=\dim(V)=n$$ $$\implies n-\text{row rank}+\text{column rank}=n$$ $$\implies \text{column rank}=\text{row rank}$$

$\endgroup$
  • $\begingroup$ This is maybe the most pedagogical one of the answers. You need of course a separate proof of the rank-nullity theorem, but this shows that the rank result is intimately related, in fact equivalent, to that theorem which student should know about anyway. I would maybe add that the dimension $n$ of the space of departure is the number of columns of the matrix, and that $\ker(\phi)$ is by definition the solution set to the homogeneous system $Ax=0$ associated to $A$. If found after row-reducing $A$, $\dim\ker(\phi)$ is the number of columns without pivot, which is $n$ minus the row rank. $\endgroup$ – Marc van Leeuwen Sep 27 '16 at 16:31
  • 1
    $\begingroup$ Why is $\dim\ker\phi = n - \text{row rank}$? I found it easier to relate row rank to $(\ker\phi^T)^\perp$, so I end up passing through the dimension of the codomain $m=\dim W$ with $\text{row rank}=m-(m-r)$, rather than using the dimension of the domain $n=\dim V$ to get $\text{row rank}=n-(n-r)$ as you have done. $\endgroup$ – ziggurism Feb 22 '17 at 19:22
  • $\begingroup$ @ziggurism If you see the Matrix as a linear system the dimension of its kernel is the number of "useless" equations, if you understand what I mean. $\endgroup$ – Lonidard Feb 22 '17 at 19:24
  • $\begingroup$ I guess it can go two ways: 1. relate the rowspace $\im\phi^T$ to the kernel $\ker\phi\cong(\im\phi^T)^\perp$ by orthogonal complement in the domain so that $\text{row rank}=n-\dim\ker\phi$ and then first isomorphism theorem gives $V/\ker\phi\cong \im\phi$ so that $\text{row rank}=n-(n-r)=r$, or else 2. relate the rowspace $\im\phi^T$ in the domain to orthogonal complement of the kernel of the transpose $(\ker\phi^T)^\perp$ in the codomain by the first isomorphism theorem, and thence to the complementary space $\ker\phi^T\cong(\im\phi)^\perp,$ from which we have $\text{row rank}=m-(m-r)$. $\endgroup$ – ziggurism Feb 23 '17 at 3:47
  • $\begingroup$ So you chose the first option, and I landed on the second option. Of course the two options are equivalent, for reasons which I suppose boil down to OP's question. $\endgroup$ – ziggurism Feb 23 '17 at 3:49
10
$\begingroup$

I think Strang's "four subspaces" picture is enlightening here. Assume $A \in \mathbb R^{m \times n}$. It's easy to prove that the null space of $A$ and the image of $A^T$ are orthogonal complements. Also, $A$ (as a mapping) is one to one when restricted to the range of $A^T$. So the range of $A^T$ is actually isomorphic to the range of $A$, and $A$ itself provides the isomorphism!

Strang presents the four subspace picture in the context of inner product spaces (in fact just $\mathbb R^n$). But the picture works for arbitrary finite dimensional vector spaces if we use annihilators instead of orthogonal complements. This gives an easy, conceptual proof that $A$ and $A^T$ have the same rank. See chapter two of Lax's linear algebra book for the (easy) details.

$\endgroup$
6
$\begingroup$

An alternative 'symmetric' definition for the rank might help here.

Define the rank of $A \in F^{m\times n} $ as the minimal $k$ for which $A$ can be represented as $$ A = c_1 r_1 + \cdots + c_k r_k$$ where $c_i \in F^{m\times 1}$ and $r_i \in F^{1 \times n}$.

Then it follows that each row of $A$ is a linear combination of $r_1, \cdots, r_k$ since the $i$th row of $A$ will be $(c_1)_i r_1 + \cdots + (c_k)_i r_k$. This implies $\text{row rank} \le k$

On the other hand, if we let $r'_1, \cdots, r'_t$ be a basis of the row space, we can write each of row of $A$ as a linear combination of $r'_i$s. Then by collecting the coefficients of $r'_i$ and making it a column vector $c'_i$, we have $A = c'_1 r'_1 + \cdots + c'_t r'_t$. Thus $\text{row rank} = t \ge k$ by the minimality of $k$.

Summing these up, we have $k = \text{row rank}$. And with the same arguments, once can easily prove $k = \text{column rank}$.

$\endgroup$
4
$\begingroup$

Let $V$ be a vector space of dimension n, W of dimension $m$. If $M$ is m by n matrix, it gives a linear map from $V$ to $W$. The image of this map is the column space. The dual map $W^* \rightarrow V^*$ is given by the transpose $M^T$. The image of this map is therefore the row space. Further, the Hom functor is exact in this context, so the exact sequence: $$0 \rightarrow ker(M) \rightarrow V \overset M \rightarrow M(V) \rightarrow 0$$ gives an exact sequence:

$$ 0 \rightarrow M(V)^* \overset {M^T|_{M(V)^*}} \rightarrow V^* \rightarrow ker(M)^* \rightarrow 0$$

M(V) has the same dimension as $M(V)^*$. By exactness of Hom again, the image of the map $M^T|_{M(V)^*}$ (which is $M(V)^*$) coincides with the image of $M^T$, the row space.

Note: I think only left exactness of Hom is used here...

$\endgroup$
3
$\begingroup$

Given a linear system of equations:$$\begin{cases}ax+by = c_1\\ cx+dy = c_2\end{cases}$$ Is there a solution?

One way to answer this is; if the lines are not parallel, then there is a solution. From the slope-intercept form of the equation of a line: $y=mx+b$, $y=-\dfrac abx + \dfrac{c_1}b$ and $y=-\dfrac cdx + \dfrac{c_2}b$.

If the lines are to intersect, the slopes can not be equal: that is $\dfrac ab \neq \dfrac cd$, which gives: $ad \neq bc\implies ad-bc \neq0$ ....the determinant can not equal zero.

Alternatively, solving for $x$ instead: $x=-\dfrac ba + \dfrac{c_1}a$ and $x =-\dfrac dc + \dfrac{c_2}c$.

$\dfrac ba \neq \dfrac dc$ and again: $da-bc \neq 0$

You can also see that the coefficients of $x$ and $y$ are "intimately" related.

I doubt the above is rigourous enough for most people, but this helps me see the relationship between rows and columns $\ldots$ between coefficients.

$\endgroup$
3
$\begingroup$

This can also be understood in terms of the singular value decomposition. Although proving the SVD takes a bit of work, once proved it provides a more thorough intuition about what is going on.

Geometrically, the SVD says that a matrix maps the unit (hyper-)sphere in the domain to a (hyper-)ellipsoid in the range, possibly squashing some axes of the ellipsoid to be flat. Moreover, the axes of the ellipsoid in the range correspond to an orthonormal set of vectors on the sphere in the domain.

The non-flat axes of the ellipsoid span the column space, and their corresponding axes on the sphere span the row space. Since these axes are orthonormal and in one-to-one correspondence, the fact that their dimension is the same becomes trivial.

$\endgroup$
  • $\begingroup$ Also see this excellent answer which states the same in a bit more detail $\endgroup$ – akraf Nov 14 '17 at 16:54
2
$\begingroup$

Warm-up

Ok. The best way is to take a concrete matrix. Take any matrix without loss of generality.

$$ A = \begin{pmatrix} 3 & 12 & 10 & 3 \\ 14 & 3 & 13 & 7 \\ 5 & 15 & 4 & 9 \end{pmatrix} $$

If you look at this matrix, you notice that in its columns you have 4 tuples of real numbers, each of which can be represented as points in $\mathcal{R}^3$. First column is a tuple $(3,14,5)$, a point in 3D-space.

So the column rank in our case cannot be more than 3 because we use 3 values to locate all points in 3D-space. Same goes for the rows, and the row rank of this matrix cannot be more than 4. It can be 4, but no more.

$$ $$

Some rows are not independent

Now imagine that without loss of generality you could represent the first row as a linear combination of other rows, e.g.

$$R_1 = aR_1 + b R_2$$

You fix these values $a,b$ mentally and see what they mean for the columns of $A$.

$$ $$

Translation into column space

They mean that in each column every top cell is a linear combination of all other values in the same column (with the same values $a,b$). That is

Value 3 is a linear combination of 14 and 5. Value 12 is a linear combination of 3 and 15.

Value 10 is a linear combination of 13 and 4. Value 3 is $7a+9b$.

$$$$ Conclusion

The main point is that we can do linear combinations of rows and columns with the same scalars $a,b$. So the column rank of our matrix would be 2. In our case the first coordinate of our points in $\mathcal{R}^3$ is redundant.

The rank of our matrix is the smallest of the column rank and the row rank.

$\endgroup$
2
$\begingroup$

I was brought here by a comment on this question, but while many of the answers are very good, none of them quite address how I think about this question.

That said, this answer is most similar to littleO's answer, but it's a different perspective. As a geometer, I like to think about the equations that vanish on a space.

Thus if I have a linear map $T : V\to W$, we note that it induces a map $T^*:W^*\to V^*$, and the kernel of $T^*$ is precisely the set of linear functionals on $W$ that vanish on the image of $V$. The dimension of a subspace generally is the dimension of the ambient space minus the number of equations needed to cut out the subspace. Thus $$\newcommand\im{\operatorname{im}}\dim\im T = \dim W-\dim\ker T^*.$$ By rank nullity, we can relate the dimension of the kernel of $T^*$ to the rank of $T^*$, which is the row rank of $T$ to get, $$\dim\im T^*=\dim W-\dim \ker T^*,$$ so $$\dim\im T=\dim\im T^*,$$ or row rank equals column rank.

$\endgroup$
1
$\begingroup$

Maybe this helps a bit with the intuition: When you transpose a matrix you don't change the dimension of the image. But when you transpose a matrix, the column rank becomes the row rank and vice versa. As the dimension of the image is the column rank those are equal.

$\endgroup$
  • 8
    $\begingroup$ Saying that when you transpose a matrix you don't change the dimension of the image is just saying that the transpose has the same column rank as the original. How is this easier to see that that row and column rank are the same? $\endgroup$ – Marc van Leeuwen Sep 16 '13 at 7:22
  • 1
    $\begingroup$ @MarcvanLeeuwen well sometime just rephrasing a statement makes it easier to see $\endgroup$ – Dominic Michaelis Sep 16 '13 at 9:16
  • 5
    $\begingroup$ Agreed, but I don't see how that would apply to the current case. $\endgroup$ – Marc van Leeuwen Sep 16 '13 at 9:22
0
$\begingroup$

To see this without calculation, I guess the key is to realize that the dual basis and basis of column space of a matrix are one to one relationship. Therefore the ranks are the same.

$\endgroup$
  • 2
    $\begingroup$ What's that, the (dual) basis of a matrix? This seems nonsense to me. $\endgroup$ – Marc van Leeuwen Sep 27 '16 at 16:25

protected by Saad Oct 5 '18 at 4:07

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.