4
$\begingroup$

A matrix can be thought of in terms of columns (then it represents the basis vectors of an coordinate system) or in terms of rows (then it represents a set of linear equations).

How can we combine those two views?

Example: Let the matrix $A$ represent a set of linear equations. Then each row is a linear equation. But we can also see the matrix as the basis vectors of a coordinate system. What coordinate system does matrix $A$ represent? What is the link between the column-wise and row-wise viewpoints on matrix?

$\endgroup$
4
$\begingroup$

One way to relate the row and column spaces of a matrix is via the idea of a dual space.

If $V$ is a vector space over $\mathbb K$, then its dual space $V^*$ consists of all linear functionals $\mathbf\alpha:V\to\mathbb K$. It’s easy to verify that $V^*$ is also a vector space and that for finite-dimensional spaces, $\dim V=\dim V^*$. If $(\mathbf{e}_1,\dots,\mathbf{e}_n)$ is a basis for $V$, the dual basis of $V^*$ is a basis $(\mathbf{\epsilon}^1,\dots\mathbf{\epsilon}^n)$ such that $\mathbf{\epsilon}^i[\mathbf{e}_j]=\delta_{ij}$. If we represent elements $\mathbf v$ of $V$ as column vectors relative to some basis, then we can use row vectors relative to the dual basis to represent elements $\mathbf\alpha$ of $V^*$, and $\mathbf\alpha[\mathbf v]$ can be computed via matrix multiplication. (Since $(\mathbb R^n)^*$ is isomorphic to $\mathbb R^n$, the distinction between them is often glossed over.) For the rest if this answer, I’ll take $\mathbb K=\mathbb R$, since that’s likely the most familiar to you.

Let $V=\mathbb R^m$ and $W=\mathbb R^n$ and $L:V\to W$ a linear map. We know that, relative to bases $(\mathbf{v}_1,\dots,\mathbf{v}_m)$ and $(\mathbf{w}_1,\dots,\mathbf{w}_n)$ of $V$ and $W$, respectively, $L$ can be represented by some $m\times n$ matrix $A$. If $\mathbf v=\sum_{i=1}^mc_i\mathbf{v}_i$, then $A\mathbf v=\sum_{i=1}^mc_iA\mathbf{v}_i$. This means that the columns of $A$ are the images of the basis vectors $\mathbf{v}_i$ and that $\operatorname{im}L$ is spanned by the columns of $A$.

Looking at the rows of $A$ instead, with the above interpretation of the product of a row and column vector, we can treat each row of $A$ as an element $\mathbf{\alpha}^i$ of $V^*$. If $L\mathbf v=\sum_{i=1}^nd_i\mathbf{w}_i$, then the rules of matrix multiplication tell us that $d_i=\mathbf{\alpha}^i[\mathbf v]$. For a vector $\mathbf v\in\ker L$, this gives us a system of equations $$\begin{align}\mathbf{\alpha}^1[\mathbf v]&=0\\\vdots\\\mathbf{\alpha}^m[\mathbf v]&=0\end{align}$$ satisfied by $\mathbf v$. By linearity, for all $\mathbf{\alpha}\in\operatorname{span}{\{\mathbf{\alpha}^1,\dots,\mathbf{\alpha}^m\}}$, we also have $\mathbf\alpha[\mathbf v]=0$. A functional that maps every element of some space $V$ to zero is said to annihilate that space, and the space of linear functionals that annihilate $V$ is called the annihilator of $V$, commonly denoted by either $V^0$ or $V^\perp$. So, the row space of $A$ annihilates $\ker L$.

To complete the connection, we need the $adjoint$ of $L$. This is the linear operator $L^*:W^*\to V^*$ (note the reversal of direction) such that for $\mathbf\beta\in W^*$ and $v\in V$, $(L^*\mathbf\beta)[\mathbf v]=\mathbf\beta[L\mathbf v]$. If $A$ is the matrix of $L$ relative to some choice of bases for $V$ and $W$, then the matrix of $L^*$ relative to their dual bases is simply $A^T$. The columns of $A^T$ span the image of $L^*$, but the columns of $A^T$ are the rows of $A$, so we see that $\operatorname{im}{L^*}=(\ker L)^0$. It’s also fairly easy to show that $\ker{L^*}=(\operatorname{im}L)^0$.

If we set $V=W=\mathbb R^n$ and identify this space with its dual via the isomorphism mentioned earlier, we can interpret $\mathbf\alpha[\mathbf v]$ as an inner product. The above then becomes the well-known result for a square matrix $A$: its row space is the orthogonal complement of its null space and its column space is the orthogonal complement of the null space of its transpose.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.