Demystifying the Determinant

Question

Preface: I’ve read a few of the posts on here about the determinant but none seem to put to rest the the questions I have, so I’ve decided to ask outright.

Question: In general, I would like to just have a deeper understanding of where the determinant comes from. It seems so fundamental to almost any field of mathematics. So here are my questions about it:

1. Who discovered the formula for the determinant? Was it noticed or was it built for a specific purpose? If so what was that motivation?

2. Why/how does it relate to consistent/inconsistent systems?

3. How does it relate to cross product? Why is it that two vectors aligned in a certain way produces a third orthogonal vector and how does the determinant make this happen?

4. Why is the determinant used in finding eigenvalues and eigenvectors?

Any details would be very much appreciated!

Answer 1

One of my favorite explanations is this one from Ask A Mathematician. Basically starting off of the idea of finding the volume of the parallelogram that any transformation takes the unit cube, it creates from scratch the definition of determinant. I found this extremely amazing because usually determinants are introduced in the "here's how you calculate it" way and then later shown to be the volume of the transformed unit cube, but this resource starts from the very intuitive notion of "volume of the transformed unit cube" and builds the formula from there, based on some very important but still intuitive properties. Brilliant!

This "volume of transformed unit cube" idea is why it's important in inconsistent systems and invertible transformations/matrices; if the determinant is $0$ (i.e it takes the unit cube and squishes it into a lower dimension), then that means that the transformation is not invertible (not injective) and hence may not have solutions or has infinitely many solutions.

For the cross product, this resource may be illuminating: Motivation for construction of cross-product (Quaternions?). The links therein of course also provide more information, but I personally really like the one I explicitly linked. I'm not entirely sure why determinants are involved, but understanding the formula for the cross product may help in that quest for understanding. If someone has ideas, please comment or answer! I would love to know too.

And finally for eigenvalues, we want to find vectors and eigenvalues such that $A\vec v = \lambda \vec v$, which is equivalent to finding vectors in the null space of $(A-\lambda I)$. Like I said in part $(b)$, determinants are really good at telling us if the unit cube has been squished to a lower dimension (i.e. a non-zero vector has been sent by the transformation to $0$), so it's good at telling us what values of $\lambda$ give us matrices that have a non-zero null space, and hence telling us the eigenvectors/eigenspaces corresponding to a particular $\lambda$.

I might be adding more details later, but if you have questions on what I've written so far, feel free to ask.

P.S. if you want a more formal introduction to determinants and eigenvalues, one of my favorite linear algebra books has been Linear Algebra Done Right, by Sheldon Axler. See chapter 10 for determinants and chapter 5 for eigenvalues. It really sheds light on alternate ways of thinking about these things (outside of the common approach of "here's a crap ton of matrices, do some computations with them")

P.P.S I also wonder a lot about these "origin" questions in mathematics, and if you're interested in more, I'll share some awesome stuff that I've found over eons of prowling MSE:

• Physical interpretation of Laplace transforms
• Could *I* have come up with the definition of Compactness (and Connectedness)?

The answers in all the things I've linked above are just so helpful for any student of mathematics I just couldn't resist sharing!

Answer 2

If one defines the determinant of a square matrix $M$ to be the signed volume of the result of applying $M$ to the unit cube (i.e. the region spanned by the unit vectors), then one can observe:

1. $\det(MN) = \det(M)·\det(N)$ for any square matrices $M,N$ of the same size.
2. $\det(D)$ is the product of the diagonal entries of $D$ for any diagonal matrix $D$, because $D$ is the transformation matrix of a scaling.
3. $\det(P) = 0$ for any matrix $P$ that has only $0$ in some row, because $P$ is the transformation matrix of a projection that collapses everything into some hyperplane with some coordinate zero.
4. $\det(R) = -1$ for any matrix $R$ that is obtained from $I_k$ by swapping two rows, because $R$ is the transformation matrix of a reflection that swaps two coordinate axes.
5. $\det(E) = 1$ for any matrix $E$ that has only $1$ on its diagonal and only one non-diagonal entry, because $D$ is the transformation matrix of an axis-parallel shear.

Now it is not hard to show that every matrix $M$ can be expressed (via Gaussian elimination) as a product $A_1 A_2 \cdots A_k B$ where each $A_i$ is of the form (4) or (5) above, and $B$ is of the form (2) or (3) above. Thus $\det(M)$ can be computed in the process of Gaussian elimination, and indeed is most efficiently done that way.

Moreover, if $B$ is of form (2), then $M$ is invertible, since all matrices of form (2) or (4) or (5) are invertible. But if $B$ is of form (3), then $M$ is not invertible since $B$ will map some input vectors to the same output vector (this can be easily observed from the row-echelon form of $B$). Thus $M$ is invertible iff $\det(M) ≠ 0$. Which should really be expected, since intuitively $M$ is invertible iff it does not flatten the unit cube, and once flattened no linear transformation can unflatten it. If $M$ is not invertible, then the equation $Mx = y$ may have no solution or infinitely many solutions, depending on whether the range of $M$ contains $y$ or not, respectively.

Furthermore, it is easy to check that the Leibniz definition of determinant satisfies all the above properties, and hence must be the same. The Leibniz definition can hence be motivated by the desire to have a rigorous definition that coincides with the 'intuitive geometric definition'. This is captured by the algebraic property of the determinant being an "alternating multilinear map with respect to the columns" as described by the wikipedia article. Here "multilinear" corresponds to (2)+(3)+(5), and the "alternating" qualifier corresponds loosely to (4).

The cross product is peculiar to $3$ dimensions, and celtschk has already explained how one can arrive at its relation to the determinant of $3×3$ matrices.

As for eigenvectors of a $k×k$ matrix $M$, we want to find some nonzero vector $v$ and scalar $λ$ such that $Mv = λv$, equivalently $(M-λI_k)v = 0$, which requires $(M-λI_k)$ to be not invertible, which as explained above is equivalent to $\det(M-λI_k) = 0$.

Answer 3

The beautiful interpretation of the determinant as a signed volume gets so much attention that sometimes I worry new students won't realize that the determinant can easily be discovered just by solving $Ax = b$ by hand, using high school algebra, in the case where $A$ is a $2 \times 2$ or $3 \times 3$ matrix. For example, if $A$ is $2 \times 2$, we want to solve \begin{align} a_{11} x_1 + a_{12} x_2 &= b_1 \\ a_{21} x_1 + a_{22} x_2 &= b_2. \end{align} Multiplying both sides of the the first equation by $a_{21}$, and then multiplying both sides of the second equation by $a_{11}$ and subtracting, we find that $$ (a_{11} a_{22} - a_{21} a_{12}) x_2 = a_{11} b_2 - a_{21} b_1. $$ If the number $a_{11} a_{22} - a_{21} a_{12}$ is not zero, then we can solve for $x_2$. (And we find a similar formula for $x_1$.)

So immediately, we have discovered the determinant of a $2 \times 2$ matrix, and we see that if this special number is not zero then $Ax = b$ has a unique solution for any choice of $b$. The case where $A$ is $3 \times 3$ is not much harder. Someone who is sufficiently obsessed, as many mathematicians are, would not hesitate to work out the $4 \times 4$ case by hand if the pattern is not already clear. So, there's nothing difficult about discovering the determinant. (By the way, we have also discovered Cramer's rule in the process.)

---

If $\lambda$ is an eigenvalue of $A$, this means that there exists some nonzero vector $x$ such that $Ax = \lambda x$. Equivalently, $$ \tag{1} (A - \lambda I) x = 0, $$ where $I$ is the identity matrix.

But, as we have discovered above, if $\det (A - \lambda I) \neq 0$, then (1) has a unique solution, which is $x = 0$. So the fact that there is a connection between eigenvalues and the determinant is immediate. You might conjecture that if $\det (A - \lambda I) = 0$ then (1) has a nonzero solution, and it turns out to be possible to show that this is correct. So, $\lambda$ is an eigenvalue of $A$ if and only if $\det (A - \lambda I) = 0$.

Answer 4

On the relation of the determinant and the cross product:

Start with the fact that the volume of the three-dimensional parallelepided spanned by the vectors $\vec a$, $\vec b$, $\vec c$ is given by the determinant $$V = \det\begin{pmatrix} a_1 & b_1 & c_1\\ a_2 & b_2 & c_2\\ a_3 & b_3 & c_3 \end{pmatrix}$$ Now do a minor expansion on the first column: $$V = a_1 \det\pmatrix{b_2 & c_2\\b_3 & c_3} - a_2 \det\pmatrix{b_1 & c_1\\b_3 & c_3} + a_3 \det\pmatrix{b_2 & c_2\\b_3 & c_3}$$ If you look closer to this formula, it looks like the scalar procuct of the vector $\vec a$ with the vector $$\begin{pmatrix} \det\pmatrix{b_2 & c_2\\b_3 & c_3}\\ -\det\pmatrix{b_1 & c_1\\b_3 & c_3}\\ \det\pmatrix{b_2 & c_2\\b_3 & c_3} \end{pmatrix} = \begin{pmatrix} b_2 c_3-b_3 c_2\\ b_3 c_1-b_1 c_3\\ b_1 c_2-c_2 b_1 \end{pmatrix}$$ Now if we look at this term, we notice that it is linear in both $\vec b$ and $\vec c$, that is, it has the properties of a product. Therefore we can introduce this as a new product, $\vec b\times\vec c$. The properties of that product are then easily derived from the properties of the determinant:

If $\vec a$ is in the plane spanned by $\vec b$ and $\vec c$, the determinant is zero. That means, $\vec a\cdot(\vec b\times\vec c)$ is zero. And that in turn means that $\vec a$ is orthogonal to $\vec b\times\vec c$. Or in other words, $\vec b\times\vec c$ is orthogonal to the plane spanned by $\vec b$ and $\vec c$.

If $b$ and $c$ are linearly dependent (that is, multiples of each other), then the determinant is $0$, no matter what $\vec a$ is. But the only vector where the scalar product with all vectors gives zero is the zero vector. Thus $\vec b\times\vec c=\vec 0$ whenever $b$ and $c$ are linearly dependent.

Moreover, exchanging the two factors means exchanging the two columns in the matrix, which means changing the sign of the determinant. Therefore $\vec c\times\vec b=-\vec b\times\vec c$.

Also the fact that $\vec a\cdot(\vec b\times\vec c)=\vec b\cdot(\vec c\times\vec a)=\vec c\cdot(\vec a\times\vec b)$ directly follows from the invariance of the determinant under even-parity permutations of the columns.

Answer 5

1. See https://en.wikipedia.org/wiki/Determinant#History.

2. You can understand a determinant as the measure of the volume defined by its column (or row) vectors. In 2D, two vectors define a parallelogram; in 3D, three vectors define a parallelepiped; in N dimensions, you can say parallelotope. The determinant is zero iff the vectors are linearly dependent (hence the volume is flat). Linear dependence is the property that lets you discuss system consistency. In particular, $M\textbf x=0$ only has non-trivial solutions ($\mathbf x\ne0$) when the vectors are linearly dependent, i.e. whe the system determinant is zero, and conversely.

3. The cross-product is in fact a pseudo-determinant as three of its elements are basis vectors instead of numbers. You understand the orthogonality property by taking the dot product with either original vectors, noticing that in general* $$\textbf a\cdot(\textbf b\times\textbf c)=(a_x\textbf i+a_y\textbf j+a_z\textbf k)\cdot\begin{vmatrix}\textbf i&\textbf j&\textbf k\\b_x&b_y&b_z\\c_x&c_y&c_z\end{vmatrix}=\begin{vmatrix}a_x&a_y&a_z\\b_x&b_y&b_z\\c_x&c_y&c_z\end{vmatrix}.$$ When two vectors are identical, the determinant is zero and so is the dot product, hence the cross-product is orthogonal to both original vectors. $\textbf b\cdot(\textbf b\times\textbf c)=\textbf c\cdot(\textbf b\times\textbf c)=0$.

4. An Eigenvector is such that it is linearly dependent with its image by the linear transform defined by the matrix ($M\textbf v=\lambda\textbf v$ or $(M-\lambda I)\textbf v=0$). So refer to 2.

---

*You obtain this result by developing the determinant on its first row and computing the dot product. This is the same as copying $\textbf a$ to the first row.