Visulizing column/row space and null/left null space, A and x

Question

So here is a direct passage from the book we are learning from.

Every vector $\boldsymbol x$ in the nullspace is perpendicular to every row of $A$, because $A\boldsymbol x=\boldsymbol 0$. The nullspace $N(A)$ and the row space $C(A^{\rm T})$ are orthogonal subspaces of $\,{\bf R}^n$.

To see why $\boldsymbol x$ is perpendicular to the rows, look at $A\boldsymbol x=\boldsymbol 0$. Each row multiplies $\boldsymbol x$:

$$A\boldsymbol x=\begin{bmatrix}\text{row }1\\\vdots\\\ \quad\!\text{row n}\quad \end{bmatrix}\begin{bmatrix}\\ \boldsymbol x \\ \,\,\end{bmatrix}=\begin{bmatrix}0\\\vdots\\0\end{bmatrix}\quad\matrix{\longleftarrow&\; \text{(row 1)}\cdot\boldsymbol x\text{ is zero}\\ \\ \longleftarrow&\; \text{(row $m$)}\cdot\boldsymbol x\text{ is zero} } \tag1$$

Now the image I get in my mind is that A belongs to column space, A(transpose) belongs to row space and that x belongs to null space/left nullspace depending on we are calculating column or row space. This is the books best description of what represents what. I am trying to get an image of my head how things are related.

The image below is what I'm trying to figure out, x is in nullspace/left nullspace and the vectors a1 and a2 that belong to Matrix A is in rowspace and columnspace. Is this an accurate thinking of how the vectors are represented when x is nullspace/left nullspace if trying to visualize them?

Answer 1

There is no point in trying to visualise this since the row space of $A$ and its null space are not naturally subspaces of the same space. If $A$ corresponds to a linear map $V\to W$, then the null space of $A$ corresponds to the kernel $\ker(f)$, a subspace of$~V$. Each row of $A$ computes one coordinate of the image of $f$, so it is a linear form on $V$ (a linear function $V\to\Bbb R$). The row space is defined by the linear combinations of these rows, so it is naturally a subspace of the space $V^*$ of linear forms on$~V$. This space $V^*$ not the same space as$~V$, even though it has the same dimension; it is called its dual vector space. The row space is then a subspace $R\subseteq V^*$, and the natural statement of what the cite passage is saying is that $\ker(f)$ is the set of vectors where all linear forms of$~R$ are zero simultaneously. This is quite unsurprising, since the null $\ker(f)$ is by definition the set where the linear forms corresponding to the rows of $A$ all are zero (the intersection of their zero sets), but once this happens for some $v\in V$, any linear combination of those linear forms is also zero at$~v$.

Now imagining combinations of linear forms on $V$ might be a bit hard, so one may just imagine that $V$ has an inner product for which the basis that one was using (to define the matrix $A$ of $f$) is orthonormal. This inner product is artificial and has no clear meaning, but at least it allows to represent each linear forms as the inner product with one specific vector$~v_i$. This representing of linear forms by vectors is what goes on in the operation of transposition. Then row$~i$ of $A$ corresponds to (the inner product with)$~v_i$, and the row space corresponds to the span of $v_1,\ldots,v_m$. Now the subspace where the linear form for row$~i$ of$~A$ is zero is the set of vectors orthogonal to$~v_i$. Then $\ker(f)$ is the set of vectors where this happens for all rows at once, i.e., the subspace of vectors orthogonal to $v_1,\ldots,v_m$, which is the orthogonal complement of the space of $v_1,\ldots,v_m$. This is a way to visualise, but remember that we are just saying that all linear forms in $R$ are zero simultaneously.

Answer 2

let us visualize the null space of a matrix in three dimensions. suppose $\vec n = (1,2,3)^T$ is in the null space of $A$. this is a line, can you visualize the plane of vectors $\vec x^T = (x,y,z)$ that is orthogonal to $\vec n.$ the components of $\vec x $ satisfy $0 = \vec x . \vec n = \vec x^T \vec n = x + 2y + 3z.$ the rows of the matrix are all made of vectors like $\vec x.$ this is a way of seeing the fact that the row space of $A$ is orthogonal to the null space of $A.$