67
$\begingroup$

I am currently trying to self-study linear algebra. I've noticed that a lot of the definitions for terms (like eigenvectors, characteristic polynomials, determinants, and so on) require a square matrix instead of just any real-valued matrix. For example, Wolfram has this in its definition of the characteristic polynomial:

The characteristic polynomial is the polynomial left-hand side of the characteristic equation $\det(A - I\lambda) = 0$, where $A$ is a square matrix.

Why must the matrix be square? What happens if the matrix is not square? And why do square matrices come up so frequently in these definitions? Sorry if this is a really simple question, but I feel like I'm missing something fundamental.

$\endgroup$
  • 7
    $\begingroup$ Note that you can't compute the determinant of a non-square matrix. $\endgroup$ – Arnaud Mortier Jun 7 '18 at 23:53
  • 10
    $\begingroup$ @ArnaudMortier To be fair, I think that will just constitute another instance of the OP's question: "Why does the determinant only make sense for square matrices?" $\endgroup$ – Noah Schweber Jun 7 '18 at 23:55
  • 1
    $\begingroup$ One rather successful attempt to bring much of the machinery of square matrices to $m\times n$ matrices is the singular value theorem. It does so by doing the spectral decomposition of $A^tA$ which is also symmetric $\endgroup$ – N8tron Jun 8 '18 at 11:33
  • $\begingroup$ Note, the definitions of eigenvectors etc that you mention don't depend on the matrix having real values. Matrices with complex values are just as useful (and of course a real matrix can be considered as a special case of a complex matrix). Also (as N8tron said) there is a very powerful concept that applies to non-square matrices, namely the "singular value decomposition" (SVD) - though to understand what it is and what it's good for, you need to learn about the concepts you mentioned for square matrices first, and the SVD may not be covered at all in a "first" linear analysis course. $\endgroup$ – alephzero Jun 10 '18 at 21:08
135
$\begingroup$

Remember that an $n$-by-$m$ matrix with real-number entries represents a linear map from $\mathbb{R}^m$ to $\mathbb{R}^n$ (or more generally, an $n$-by-$m$ matrix with entries from some field $k$ represents a linear map from $k^m$ to $k^n$). When $m=n$ - that is, when the matrix is square - we're talking about a map from a space to itself.

So really your question amounts to:

Why are maps from a space to itself - as opposed to maps from a space to something else - particularly interesting?

Well, the point is that when I'm looking at a map from a space to itself inputs to and outputs from that map are the same "type" of thing, and so I can meaningfully compare them. So, for example, if $f:\mathbb{R}^4\rightarrow\mathbb{R}^4$ it makes sense to ask when $f(v)$ is parallel to $v$, since $f(v)$ and $v$ lie in the same space; but asking when $g(v)$ is parallel to $v$ for $g:\mathbb{R}^4\rightarrow\mathbb{R}^3$ doesn't make any sense, since $g(v)$ and $v$ are just different types of objects. (This example, by the way, is just saying that eigenvectors/values make sense when the matrix is square, but not when it's not square.)


As another example, let's consider the determinant. The geometric meaning of the determinant is that it measures how much a linear map "expands/shrinks" a unit of (signed) volume - e.g. the map $(x,y,z)\mapsto(-2x,2y,2z)$ takes a unit of volume to $-8$ units of volume, so has determinant $-8$. What's interesting is that this applies to every blob of volume: it doesn't matter whether we look at how the map distorts the usual 1-1-1 cube, or some other random cube.

But what if we try to go from $3$D to $2$D (so we're considering a $2$-by-$3$ matrix) or vice versa? Well, we can try to use the same idea: (proportionally) how much area does a given volume wind up producing? However, we now run into problems:

  • If we go from $3$ to $2$, the "stretching factor" is no longer invariant. Consider the projection map $(x,y,z)\mapsto (x,y)$, and think about what happens when I stretch a bit of volume vertically ...

  • If we go from $2$ to $3$, we're never going to get any volume at all - the starting dimension is just too small! So regardless of what map we're looking at, our "stretching factor" seems to be $0$.

The point is, in the non-square case the "determinant" as naively construed either is ill-defined or is $0$ for stupid reasons.

$\endgroup$
  • 3
    $\begingroup$ I think your answer is better than mine. $\endgroup$ – Ethan Bolker Jun 7 '18 at 23:58
  • 4
    $\begingroup$ I'd suggest that (as offered up passingly in gandalf61's answer) another piece of the puzzle is that once you have the set of $n\times n$ matrices, there's also a natural composition operator on this set, one that can take any two members and produce a third; this isn't the case for arbitrary-dimensioned matrices, and this additional structure brings along with it all the natural questions that you can ask when you have a binary operator on a set. $\endgroup$ – Steven Stadnicki Jun 8 '18 at 22:37
  • 3
    $\begingroup$ Fascinating! I probably would have enjoyed linear algebra a lot more if it had been accompanied with these sorts of explanations. $\endgroup$ – ApproachingDarknessFish Jun 10 '18 at 23:19
18
$\begingroup$

Lots of good answers already as to why square matrices are so important. But just so you don't think that other matrices are not interesting, they have analogues of the inverse (e.g., the Moore-Penrose inverse) and non-square matrices have a singular-value decompition, where the singular values play a role loosely analogous to the eigenvalues of a square matrix. These topics are often left out of linear algebra courses, but they can be important in numerical methods for statistics and machine learning. But learn the square matrix results before the fancy non-square matrix results, since the former provide a context for the latter.

$\endgroup$
16
$\begingroup$

In linear algebra matrices usually represent linear transformations between vector spaces. An $m \times n$ matrix $M$ represents a linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$

When the matrix is square ($m=n$) it can be thought of as representing a transformation from $\mathbb{R}^n$ to itself. That's when the concepts of eigenvalues and the characteristic polynomial make sense.

$\endgroup$
12
$\begingroup$

To add to @Ethan's answer what is fundamental when a linear map goes from a vector space to itself is that you can compare a vector with its image:

  • was it rotated?
  • was its length scaled?
  • was it turned into a multiple of itself?

Or a set of vectors with its image:

  • by how much was the volume of this parallelepiped stretched?

All these questions make sense (and carry an awful lot of information on the linear map) only when the domain and codomain coincide. The determinant, characteristic polynomial, etc. are there to answer these questions.

$\endgroup$
9
$\begingroup$

One point not made explicit in the answers so far is that powers of a matrix only make sense if the matrix is square.

A square matrix $\mathcal{M}(A)$ represents a linear map $A:V \rightarrow W$ for which the dimensions of its domain $V$ and its codomain $W$ are the same, so its domain and its codomain are isomorphic as vector spaces (often we think of the domain and codomain as being the same vector space, but strictly speaking this is not necessary).

If the domain and codomain of $A$ are the same vector space then we can compose $A$ with itself, creating linear maps $A \circ A$, $A \circ A \circ A$ etc. These are in turn represented by the square matrices $\mathcal{M}^2$, $\mathcal{M}^3$ etc. This allows us to define polynomial functions of a square matrix, and hence to make sense of statements such as the Cayley-Hamilton theorem.

$\endgroup$
  • $\begingroup$ "If the domain and codomain of $A$ are isomorphic as vector spaces then we can compose A with itself" - This is not the case. The domain and codomain have to be equal for you to compose the transformation with itself. While a square matrix might sometimes represent a map between distinct spaces of the same dimension, everything that the question asks about are intrinsic properties of maps from a space to itself, so it doesn't feel particularly relevant to me to bring up a different semantic case where determinants, characteristic polynomials, and eigenvectors don't naturally apply. $\endgroup$ – Milo Brandt Jun 8 '18 at 14:53
  • $\begingroup$ Yes you are right - I have edited my answer. $\endgroup$ – gandalf61 Jun 8 '18 at 15:55
7
$\begingroup$

Well a simple answer that I'm not sure anyone's given yet is just to think of each row of matrix as the coefficients of the unknowns in a set of simultaneous equations.

A square matrix has the same number of equations as unknowns (assuming none of the rows or columns are just multiples of each other) and therefore represents the left-hand side of a set of equations that has a good chance of having a unique solution.

With matrices it's often helpful to think of them geometrically but also often to think about them as equation coefficients.

$\endgroup$
1
$\begingroup$

For a matrix $\mathbf{A}$ to be invertible it has to be an $n\times n$ square matrix with there existing an $n\times n$ square matrix $\mathbf{B}$ such that, by the operations of matrix multiplication, we have

$$\mathbf {AB} =\mathbf {BA} =\mathbf {I}_{n}\tag{1}$$ where $\mathbf {I}_{n}$ denotes the $n\times n$ identity matrix. This matrix $\mathbf{B}$ is then unique and is termed the inverse of $\mathbf{A}$, denoted by $\mathbf{A}^{-1}$. We see by (1) the matrices have to be square, else they would not commute. A square matrix has no inverse if and only if its determinant is $0$ and is then termed singular.

An important outcome of having invertible square matrices is that a group structure may be imposed on them. The set of all $n\times n$ invertible matrices, over some field $F$, forms a group under matrix multiplication termed the general linear group of degree $n$, or $\text{GL}(n,F)$, these being the set of nonsingular $n\times n$ matrices. It can be seen the product of two invertible matrices is invertible, and that for an invertible matrix $(\mathbf{A}^{-1})^{-1}=\mathbf{A}$.

The special linear group, $\text{SL}(n,F)$, is the subset of $\text{GL}(n,F)$ whose matrices have determinant $1$, and is a normal subgroup of it.

Consider $\text{SL}(n,\mathbb{R})$: due to its matrices having determinant $1$, the linear maps from $\mathbb{R}^n\rightarrow\mathbb{R}^n$ under it are transformed with volume and orientation preserved.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.