Understanding Linear Algebra Geometrically - Reference Request

I know geometry and I know linear algebra but when I understand a linear algebraic concept geometrically, my head just explodes and things just become so much clearer and easier to understand...not to mention easier to remember or figure out its properties and explain them to others.

Here are a few examples.

1. Orthogonal matrices - If you think of an orthogonal matrix as a rotation then some of its properties are obvious. Orthogonal matrices are always invertible because rotations can simply be reversed. They always preserve the Euclidean norm because rotating a vector doesn't change its length. Orthogonal matrices forming a group is also easy to see because it is easy to see them satisfying the group axioms.

2. Determinant - The determinant of a linear transformation can be understood as follows. Start with the (chosen) basis of your domain. It forms a parallelepiped. Call it $P$. It has a certain volume $V(P)$. Now apply your linear transformation $T$ to the chosen basis. A new parallelepiped $T(P)$ is formed and its volume in the range space (embedded in the codomain) is now $V(T(P))$. The determinant (in absolute value) is the ratio of the new volume to the old one. This intuitively explains, for example, why the determinant is zero for non-invertible transformations. The dimension of such a transformation will always be strictly less than the dimension of the domain/codomain so the volume of the transformed parallelepiped will always be zero. I always imagine a parallelepiped in $\mathbb{R}^3$ collapsing onto a plane. This also explains why the determinant of an orthogonal matrix is always $\pm1$ because rotating a parallelepiped won't change its volume. In addition, it kind of helps with the Jacobian determinant and why is the Jacobian "necessary" when transforming variables.

3. Singular value decomposition - Every matrix having an SVD says the fantastical fact that any linear transformation can be considered a rotation, then a dilation (different directions by different factors), and then a rotation again.

4. Projection matrices - Imagine an arbitrary vector's shadow onto a line or a plane. I imagine a vector collapsing onto its shadow and properties like $P^2=P$ are immediate for any projector operator $P$. Take this and run with it.

My question is, can anyone point to some good reading material where a geometric interpretation of various linear algebra concepts is offered?

This could be anyone's class/teaching notes, published papers, something from recreational mathematics, or just a good book.

• Other points ideas have interesting geometric interpretations: eigenvalues/eigenvectors, spectral theorem, positive definite matrices, matrix norms vs. spectral radius, matrices with non-negative entries. Most facts about the matrix exponential can be neatly interpreted by leveraging Lie Groups/Lie Algebras. – Omnomnomnom Mar 5 '18 at 18:39
• @Omnomnomnom I don't know too much Lie Group/Lie Algebras. Can you please add an answer clearly explaining the geometric connection with the exponential operator? – Fixed Point Mar 5 '18 at 18:41
• Reflections are not rotations by 180 degrees. – David Hill Mar 5 '18 at 18:46
• @FixedPoint I don't have the time to put a full answer together, but you might find my old answer here to be an interesting and readable introduction. – Omnomnomnom Mar 5 '18 at 18:48
• @FixedPoint so one interesting point to view through this perspective (which is particularly useful in quantum mechanics) is that the exponential of a skew-symmetric (skew-Hermitian) matrix will be an orthogonal (unitary) matrix. – Omnomnomnom Mar 5 '18 at 18:51

It might be slightly off-topic since you are searching for reading material, but the 3blue1brown YouTube channel is dedicated to this sort of geometric intuitions. More specifically, it presents abstract concept with geometric animations, spanning from neural networks to Fourier transform.

Although it is not restricted to such matter, it has an excellent playlist called Essence of linear algebra which adresses what you seek.

• Having clicked the linked YouTube channel about maths, I'm sorry for my previous comment. – GNUSupporter 8964民主女神 地下教會 Mar 6 '18 at 0:17
• While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – Xander Henderson Mar 6 '18 at 0:58
• @XanderHenderson I agree. Even if I am new to the community, I have browsed it over a few times. What suggestions do you have for me to improve the answer? Maybe a better description of the content of the channel? – Archaoss Mar 6 '18 at 3:12
• @Archaoss Exactly that. Some description of what is there, including a specific description of whatever video(s) relate to the question being asked here. – Xander Henderson Mar 6 '18 at 14:05

An intuitive view point of the Primary Decomposition Theorem for Eigenspaces is to consider an orthonormal basis. For example, for the vector space $\mathbb{R}^3$ spanned by $\left\{\mathbf{\hat{i}}, \mathbf{\hat{j}}, \mathbf{\hat{k}}\right\}$, the eigenspaces are $\mathcal{E}_1 = \mathcal{L}\left(\left\{\mathbf{\hat{i}}\right\}\right)$, $\mathcal{E}_2 = \mathcal{L}\left(\left\{\mathbf{\hat{j}}\right\}\right)$, and $\mathcal{E}_3 = \mathcal{L}\left(\left\{\mathbf{\hat{k}}\right\}\right)$.

By the Primary Decomposition Theorem for Eigenspaces, $$\mathbb{R}^3 = \mathcal{E}_1 \oplus \mathcal{E}_2 \oplus \mathcal{E}_3$$

And thus any vector must be uniquely expressed by the eigenvectors of the space, in this case $\mathbb{R}^3$. For an orthonormal basis, it is easy to see why the direct sum should span the entire space, since the eigenspaces correspond to the linear span of each basis element.

It should then be easier to have an intuition for what is happening for any eigenbasis of $V$ that is not necessarily orthonormal.

• This doesn't answer the question. – BobTheAverage Mar 5 '18 at 23:59
• @BobTheAverage, kindly look at the edit history of the question and notice how it was changed afterwards. – Xandru Mifsud Mar 6 '18 at 4:04

Here is something I use when teaching an introductory linear algebra course.

If $V$ is a vector space over a field $\mathbb{F}$, then a line in $V$ can be considered as any set of the form $$\mathcal{L} = \{ x+\alpha y \mid \alpha \in \mathbb{F}\} \subseteq V,$$ in which $x$ and $y$ are elements of $V$ (note that this includes the degenerate case of a single point if $y=0$).

If $T:V \longrightarrow W$ is a linear transformation, then $$T(\mathcal{L}) = \{ T(x)+\alpha T(y) \mid \alpha \in \mathbb{F}\}.$$ Thus, linear transformations map lines to lines. (This is the explanation I give as to why the word 'linear' is used in 'linear transformation'.)

It is worth noting that this condition is not sufficient for linearity: affine transformations of the form $x \longmapsto Ax + b$ will also map lines to lines.

The vector $y-x$ connects the end of $x$ to the end of $y$.