What is the geometric interpretation of the rowspace?

Question

I am missing something essential in my understanding of matrices as linear mappings (or matrices in general). To explain, I think I have a decent intuition for columnspace of a matrix. Basically, the columnspace can be thought of as the set of points other than the origin which can be mapped by the matrix of a linear mapping (I like to think of matrices in general as linear mappings).

What I have trouble with is the significance and geometric interpretation of the rowspace of a matrix. By the definition, I can see that the rowspace and columnspace are analagous; the rowspace is merely the columnspace of the transpose of the matrix of the linear mapping. However, I cannot explain this anymore intuitively than above. To quote Albert Einstein, "if you don't understand it simply, you don't understand it well enough". I am looking for a simple explanation (or geometric interpretation that will contribute to my imagination of linear mappings).

How would you explain this concept?

To give you some context, I ran into problems when my course started to discuss the following:

For an $n \times n$ matrix $P$, the following are equivalent:

(1) The columns of $P$ form an orthonormal basis for $\mathbb{R}^n$

(2) $P^T = P^{-1}$

(3) The rows of $P$ form an orthonormal basis for $\mathbb{R}^n$.

I understand the algebra perfectly, but I seek a level of intuition so that I would not need to fiddle around with algebra to discover these facts.

How would you explain the above concepts intuitively or geometrically, without the need for algebra to make a case?

Many thanks in advance.

Answer 1

As Ted Shifrin states in his comment to your question, the rowspace of a real matrix is the orthogonal complement of its nullspace. In fact, we have the following relationships for any (not necessarily square) real matrix $A$: $$\mathscr{C}(A^T)=\mathscr{N}(A)^\perp \\ \mathscr{N}(A^T)=\mathscr{C}(A)^\perp.$$ There is an equivalent pair of statements about a linear map, its adjoint map and annihilators of subspaces.

One consequence of these relationships is that the rowspace of a matrix is a maximal-dimension subspace of the domain for which the linear map defined by the matrix is injective. Another possible way to view the rowspace is as the space of representative elements of the quotient space $\mathbb R^m/\mathscr{N}(A)$, where $m$ is the number of columns of $A$. In light of this, the rowspace of a matrix can be considered the “natural” preimage of its column space.

Answer 2

Consider a vector

$\vec{ v}=v_x \vec{i}+v_y \vec{j}+v_z \vec{k}$

Unit vectors $\vec{i}$, ${\vec{j}}$ and $\vec{k}$ form a basis, an ordered triplet $(\vec{i}, {\vec{j}}, \vec{k})$.

What does that mean? It means they are linearly independent. But what does that mean? It means they are orthogonal. You can't form any one of them by combinig the other two in any way. This is important. In order to describe spaces of more than 1 dimension you need basis vectors. Any linearly independent set of vectors will do. But let's stick to the good old $(\vec{i}, {\vec{j}}, \vec{k})$ for now. These are intuitively simple.

And how do you form a scalar product? Scalar product is the basic concept for any vector manipulation. It's our only weapon, actually. Until tensor algebras, of course. Well, you define $\vec{i} \vec{i}=1$, $\vec{j} \vec{j}=1$ and $\vec{k} \vec{k}=1$ with any other pairing being $0$, for instance $\vec{i} \vec{j}=0$. Why? Why do we do this like this? Well, physical reasons. An object can move in one direction and in another. Then can slow down in one direction, or zig-zag or something, while still keeping the speed in the other direction constant. Speeds are independent. So we introduce objects $\vec{i}$, ${\vec{j}}$ and $\vec{k}$ so that they do not interfere one with another at all, and we model interference by the process of multiplication. Simple enough.

So, scalar product is now

$\vec{ v}\vec{u}=(v_x \vec{i}+v_y \vec{j}+v_z \vec{k})(u_x \vec{i}+u_y \vec{j}+u_z \vec{k})=v_x u_x + v_y u_y +v_z u_z$

following our little rule for basis vectors.

But look:

$\begin{pmatrix}v_x & v_y & v_z \end{pmatrix} \begin{pmatrix}u_x \\ u_y \\ u_z \end{pmatrix}=\vec{ v}\vec{u}$

So you can use matrix manipulation instead of unit vectors $(\vec{i}, {\vec{j}}, \vec{k})$. But what happened in matrix manipulation? Unit vectors, that are obviously independent, are hidden now! Do notice though that both a row vector and a column vector are just vectors. Spanned by some same unit vectors that are hidden.

How about this:

$\begin{pmatrix}v_x & v_y & v_z\\a_x & a_y & a_z \end{pmatrix} \begin{pmatrix}u_x &b_x\\ u_y&b_y \\ u_z&b_z \end{pmatrix}=\begin{pmatrix}\vec{ v}\vec{u} &\vec{ v}\vec{b}\\ \vec{ a}\vec{u}&\vec{a}\vec{b}\end{pmatrix}$

It's just a scalar product of $4$ vectors. And so on, this is how you get matrices. Matrices are just scalar products of vectors.

Now rotate your space! And stretch it! Vectors $\vec{i}$, ${\vec{j}}$ and $\vec{k}$ are no longer of unit length, nor are they along your privileged axes, but they are still orthogonal, they are still independent, it's still true that $e_ie_j=\delta_{ij}$. So, you see, any vectors obeying this can serve as basis vectors. Do notice that $\vec{i}$, ${\vec{j}}$ and $\vec{k}$ are in matrix form $(1,0,0)$, $(0,1,0)$ and $(0,0,1)$. Do notice that when rotated and stretched they're no longer in this simple form. Their mutual scalar product vanishes still though, because we just rotated them and stretched them. So any vectors obeying this can serve proudly as basis vector.

So if $\vec{ v}$ and $\vec{a}$ are independent, and additionally if their scalar product vanishes, they can certainly serve as basis vectors, and you get

$\begin{pmatrix}v_x & v_y & v_z\\a_x & a_y & a_z \end{pmatrix} \begin{pmatrix}v_x &a_x\\ v_y&a_y \\ v_z&a_z \end{pmatrix}=\begin{pmatrix}\vec{ v}\vec{v} &\vec{ v}\vec{a}\\ \vec{ a}\vec{v}&\vec{a}\vec{a}\end{pmatrix}=\begin{pmatrix}1 &0\\ 0&1\end{pmatrix}$

the identity matrix, if all vectors are of unit length.

This explains (2)!

In general basis vectors do not have to have a vanishing mutual scalar product, they only have to form a set of linearly independent vectors: you can't make any one by combining the rest through stretching and adding them. In this sense they are "orthogonal": more precisely: linearly independent. In this case, the determinant is not zero. If so, since rows and columns are just vectors, if their determinant does not vanish, they're independent women! I mean, vectors... XD

This explains (1) and (3).

Maybe this? No worries, I got a heap of reputation yesterday, I forgot I put a bounty once recently and depleted my rep, it's fine now XD

Answer 3

This is in a sense trivial but also maybe not something you'd immediately think of.

Consider a square matrix $A$ (though it doesn't have to be).

The columns are a basis for the output (column) space. The image of the cartesian grid in the input space is an affine, probably skewed (unless the matrix is orthogonal) grid in the output space.

The grid lines are parallel to these basis vectors (the columns of $A$).

Consider the image of the unit sphere in the input space. This will be some ellipsoid in the output space.

Choose a particular cartesian axis in the output space, say the z (3rd) axis.

What point on the ellipsoid image "sticks out" furthest along the z axis? i.e., what point has the greatest z coordinate?

That point will lie on the span of the vector that is the image under $A$ of the 3rd row of $A$ used as an input to $A$. i.e., the span of $A\vec{v}$ where $\vec{v}^T$ is the 3rd row of $A$.

So if the 3rd row of $A$ is $\left[a\ b\ c\right]$, then the vector in the output space whose tip is at the $\left[a\ b\ c\right]$ point of the (skewed) output space grid will run through that point of the ellipse.

One way to explain this is that the "Jacobian" of $\vec{f}(\vec{x}) = A\vec{x}$, of $\vec{f}$ with respect to $\vec{x}$, is $A$. So the direction in the input space of greatest increase of the mth output dimension (coordinate) is the mth row.

Looking at the image of the unit sphere of the input space corresponds to limiting yourself to a fixed "step size" to take along the gradient.