# What is the geometric interpretation of the transpose?

I can follow the definition of the transpose algebraically, i.e. as a reflection of a matrix across its diagonal, or in terms of dual spaces, but I lack any sort of geometric understanding of the transpose, or even symmetric matrices.

For example, if I have a linear transformation, say on the plane, my intuition is to visualize it as some linear distortion of the plane via scaling and rotation. I do not know how this distortion compares to the distortion that results from applying the transpose, or what one can say if the linear transformation is symmetric. Geometrically, why might we expect orthogonal matrices to be combinations of rotations and reflections?

• the geometry is in the the inner product say on $\mathbb{R}^n$. the transpose satisfies $\langle Ax,y \rangle=\langle x,A^ty \rangle$. orthogonal matrices satisfy $\langle x,y \rangle=\langle Ax,Ay \rangle$, they preserve the geometry.
– yoyo
Commented May 6, 2011 at 12:21
• See this answer for a geometric description of the transpose.
– Matt
Commented Jan 12, 2014 at 21:19
• Like @yoyo mentions, transpose of ${ A \in \mathbb{R} ^{m \times n} }$ is that unique ${ M }$ such that ${ \langle Ax, y \rangle = \langle x, My \rangle }$ for all ${ x \in \mathbb{R} ^{n}, y \in \mathbb{R} ^{m} }.$ (Here ${ \langle \cdot, \cdot \rangle }$ is the standard dot product, which can be motivated like here). Commented Nov 6, 2023 at 13:23

## 5 Answers

To answer your second question first: an orthogonal matrix $O$ satisfies $O^TO=I$, so $\det(O^TO)=(\det O)^2=1$, and hence $\det O = \pm 1$. The determinant of a matrix tells you by what factor the (signed) volume of a parallelipiped is multipled when you apply the matrix to its edges; therefore hitting a volume in $\mathbb{R}^n$ with an orthogonal matrix either leaves the volume unchanged (so it is a rotation) or multiplies it by $-1$ (so it is a reflection).

To answer your first question: the action of a matrix $A$ can be neatly expressed via its singular value decomposition, $A=U\Lambda V^T$, where $U$, $V$ are orthogonal matrices and $\Lambda$ is a matrix with non-negative values along the diagonal (nb. this makes sense even if $A$ is not square!) The values on the diagonal of $\Lambda$ are called the singular values of $A$, and if $A$ is square and symmetric they will be the absolute values of the eigenvalues.

The way to think about this is that the action of $A$ is first to rotate/reflect to a new basis, then scale along the directions of your new (intermediate) basis, before a final rotation/reflection.

With this in mind, notice that $A^T=V\Lambda^T U^T$, so the action of $A^T$ is to perform the inverse of the final rotation, then scale the new shape along the canonical unit directions, and then apply the inverse of the original rotation.

Furthermore, when $A$ is symmetric, $A=A^T\implies V\Lambda^T U^T = U\Lambda V^T \implies U = V$, therefore the action of a symmetric matrix can be regarded as a rotation to a new basis, then scaling in this new basis, and finally rotating back to the first basis.

• You are probably talking about the singular value decomposition ("this makes sense even if $\mathbf A$ is not square!"), not the eigendecomposition. Commented May 6, 2011 at 12:24
• Oops, I originally was going to write about the eigendecomposition and then decided to write about svd instead, but forgot to change my wording. Will edit now - thanks! Commented May 6, 2011 at 12:37
• Now, another thing: the phrase is supposed to be "if $A$ is square and symmetric..." Commented May 6, 2011 at 12:59
• Why does $V\Lambda^T U^T = U\Lambda V^T \implies U = V$? I assume it's easy to prove, as it suffices to show $\Lambda = Q\Lambda Q \implies Q = I$ for any orthogonal real matrix $Q$, but I don't quite see how to carry out the proof immediately. Commented May 14, 2018 at 20:03
• @DetachedLaconian It is actually not, there are some degrees of freedom. See math.stackexchange.com/a/644397/99220 Commented Jan 15, 2021 at 13:19

yoyo has succinctly described my intuition for orthogonal transformations in the comments: from polarization you know that you can recover the inner product from the norm and vice versa, so knowing that a linear transformation preserves the inner product ($\langle x, y \rangle = \langle Ax, Ay \rangle$) is equivalent to knowing that it preserves the norm, hence the orthogonal transformations are precisely the linear isometries.

I'm a little puzzled by your comment about rotations and reflections because for me a rotation is, by definition, an orthogonal transformation of determinant $1$. (I say this not because I like to dogmatically stick to definitions over intuition but because this definition is elegant, succinct, and agrees with my intuition.) So what intuitive definition of a rotation are you working with here?

As for the transpose and symmetric matrices in general, my intuition here is not geometric. First, here is a comment which may or may not help you. If $A$ is, say, a stochastic matrix describing the transitions in some Markov chain, then $A^T$ is the matrix describing what happens if you run all of those transitions backwards. Note that this is not at all the same thing as inverting the matrix in general.

A slightly less naive comment is that the transpose is a special case of a structure called a dagger category, which is a category in which every morphism $f : A \to B$ has a dagger $f^{\dagger} : B \to A$ (here the adjoint). The example we're dealing with here is implicitly the dagger category of Hilbert spaces, which is relevant to quantum mechanics, but there's another dagger category relevant to a different part of physics: the $3$-cobordism category describes how space can change with time in relativity, and here the dagger corresponds to just flipping a cobordism upside-down. (Note the similarity to the Markov chain example.) Since relativity and quantum mechanics are both supposed to describe the time evolution of physical systems, it's natural to ask for ways to relate the two dagger categories I just described, and this is (roughly) part of topological quantum field theory.

The punchline is that for me, "adjoint" is intuitively "time reversal." (Unfortunately, what this has to do with self-adjoint operators as observables in quantum mechanics I'm not sure.)

• I do think that it has to be proved that rotations really are rotations in two and three dimensions (i.e. finding the rotation axes etc). An important remark for the OP that is especially easy to see in your post, is, that contrary to scalings, the transpose is most naturally viewed as an application somewhere else. Commented May 6, 2011 at 16:07
• In what sense does "adjoint" intuitively correspond to "time reversal"? Would it be possible to make sense of this intuition without referring to category theory? Commented Jun 1, 2011 at 23:05
• @Elliott: that's exactly what I tried to explain in the paragraph above that, as well as the comment about Markov chains. Commented Jun 1, 2011 at 23:09
• I think what I really meant is, "I didn't understand this very well, can you please elaborate in a more elementary way?" but I'll make another attempt to understand what you said before asking you again. Commented Jun 1, 2011 at 23:17
• @Elliott: the Markov chain example can be understood as follows. Let's say we have a collection of boxes joined by tubes. In these boxes there are particles running around, and let's say each tube has a fixed direction in which it passes particles. The directions of the tubes determine a transition matrix (in a loose sense) describing the possible directions the particles can travel, and taking the transpose of this matrix corresponds to reversing the allowed direction of each tube. What does this have to do with time reversal? Well, if you ran a movie of the particles traveling in reverse... Commented Jun 1, 2011 at 23:26

Overview If a matrix A acting on vectors tells you how a vector is transformed, the matrix $$A^T$$ tells you how linear measurement of this vector are transformed.

If E is your vector space, the "space of linear measurements", or dual space is the space of all linear transformation $$E \rightarrow \mathbb{R}$$. Said differently, it is the space of all linear functions that take in a vector and output a number.

Example Let's work in $$\mathbb{R}^2$$.

• We have a generic vector $$X = \begin{pmatrix} x_1 \\ x_2 \end{pmatrix}$$
• A sample matrix $$A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$$
• A sample linear measurement $$h(X)=2x_1$$, we note it $$\begin{pmatrix} 2 \\ 0 \end{pmatrix}$$

Maybe this measurement has some meaning physically (total mass or momentum...). We want to know how the transformation A affects h.

The familiar product $$AX = \begin{pmatrix} x_1 + x_2 \\ x_2 \end{pmatrix}$$ tells you how X is transformed. Now that we have seen that X has changed we might ask, what is the new measurement of X by h? It is obtained by computing $$h(AX)$$:

$$h(Ax) = h(\begin{pmatrix} x_1 + x_2 \\ x_2 \end{pmatrix}) = 2x_1 + 2x_2$$ which we note as $$\begin{pmatrix} 2 \\ 2 \end{pmatrix}$$

So when we transform by A, h goes from $$\begin{pmatrix} 2 \\ 0 \end{pmatrix}$$ to $$\begin{pmatrix} 2 \\ 2 \end{pmatrix}$$

It turns out that this transformation is precisely what the transpose is doing, as you can verify: $$A^T\begin{pmatrix} 2 \\ 0 \end{pmatrix} = \begin{pmatrix} 2 \\ 2 \end{pmatrix}$$

This is kind of hand-wavy, if you want more details look up dual space, which makes all of this precise.

• How does this apply if $A$ is not a square matrix? i.e. Suppose $A$ sends $m$-dimensional vectors to $n$-dimensional vectors. On the other hand, $A^T$ sends $n$-dimensional vectors to $m$-dimensional vectors. Using the letters from your example, $h$ is a linear functional over an $n$-dimensional vector space, which wouldn't be able to measure $X$, which is a vector in an $m$-dimensional vector space. Commented Dec 6, 2020 at 5:26
• Yes the transpose maps a measurement on the output space to one on the input space. Maybe a way to interpret this intuitively is that A might map X into a very high dimensional space (like mapping a function to its Fourier spectrum) and we might care only about a little part of the spectrum (say the first harmonic). This little part that we care about is our h, it is a measurement on the output space. What we can do is apply A and then apply h, but that is a lot of work because we need to compute the whole spectrum. Instead we can use $A^Th$ to get the first harmonic from X directly.
– Manu
Commented Dec 14, 2020 at 4:02
• The key equality is: $A^Th(v) = h(Av), \forall v \in R^m, h \in R^{n*}$
– Manu
Commented Dec 14, 2020 at 4:03

We better interpret the geometric meaning of transpose from the view point of projective geometry. Because only in projective geometry, it is possible to interpret that of all square matrices.

It would be difficult for OP to understand and grasp the necessary premises or mathematical basis when interpreting the meaning in projective geometry of a square matrix's transpose only from a not too much lengthy answer posted here. So I would recommend you to read the article in this link first.

After read article in the above link, use the basic principles below:

1. Some basic or elementary geometric transformations are equivalent to Househodler's elementary matrices in homogeneous coordinates in projective geometry; that is what stereohomology is defined;

2. All square matrices can be represented by concatenated multiplication of elementary matrices;

3. a transpose of any elementary geometric transformation means exchanging the "center" and the "interface" of it per the definition of stereohomology.

then: the transpose of any square matrices have their own geometric meaning in projective geometry, though for complicated cases, the geometric interpretation of transpose might not be unique.

To get a geometric intuition of matrix transposition, I build upon my geometric intuition of the dot product.

The dot product seen as a projection One geometric intuition behind the dot product $$a \cdot b$$ of two vectors is to see it as follow : you first project the first vector $$a$$ on the second vector $$b$$, and then you multiply the norm of this projection by the length of the vector $$b$$.

But since the dot product is commutative, that is $$a \cdot b = b \cdot a$$, you can also see it as the multiplication of the length of $$a$$ with the length of the projection of the second vector $$b$$ on the first vector $$a$$ (you just need to mentally exchange $$a$$ and $$b$$ in the previous drawing).

So when you think of the dot product as a projection, there are $$2$$ ways to do it : either you project the first vector on the second, or you project the second vector on the first.

Link between transposition and dot product Let $$M$$ be some $$m$$ by $$n$$ matrix, $$a$$ a vector of length $$n$$ and $$b$$ a vector of length $$m$$. Let's try to find a matrix a $$n$$ by $$m$$ matrix $$X$$ such that: $$Ma \cdot b = X b \cdot a$$

There is only one solution to this equation, which is $$X = M^T$$. Let's interpret this geometrically.

The left part of this equality can be geometrically interpreted as follow :

1. Apply the linear transformation $$M$$ to the vector $$a$$
2. Project the result of this transformation on the vector $$b$$
3. Multiply the length of this projection by the length of $$b$$

The right part does the exact same thing, but by projecting $$b$$ on $$a$$ :

1. Apply some linear transformation $$X$$ to the vector $$b$$
2. Project the result of this transformation on the vector $$a$$
3. Multiply the length of this projection by the length of $$a$$

And this is where a geometric intuition comes in : if I have some transformation $$M$$ and I want to intuitively guess what will be its transposition, here are the steps I follow :

1. I picture two vectors $$a$$ and $$b$$, just as in the above drawing
2. I apply mentally apply $$M$$ that transformation to the vector $$a$$ and see how the projection (the "shadow") of the vector $$a$$ behaves.
3. I fix the vector $$a$$ and now I try to move the vector $$b$$ in such a way that the length of the projection of $$b$$ on the vector $$a$$ behaves the same way as the projection of $$a$$ on $$b$$.
4. If I can find such a transformation of $$b$$ that matches these requirements, it must be $$M^T$$ !

Let's apply this with rotations !

Time for intuition training ! Let's intuitively guess what the transposition of a rotation should be by following the previous steps :

1. Picture two vectors $$a$$ and $$b$$ like the picture above
2. Rotate the vector $$a$$ by some angle $$\phi$$ smaller than $$\theta$$ towards the vector $$b$$. The length of the projection increases.
3. Now instead of rotating $$a$$, rotate the vector $$b$$ by the same angle $$\phi$$. This should rotate $$b$$ away from the vector $$a$$. Also the length of the projection of $$b$$ on $$a$$ decreases. So this is not what we want ...
4. Let's try to rotate $$b$$ in the other direction, that is towards the vector $$a$$. Now the projection increases. It looks like the rotation $$-\phi$$ should give us what we want.

And this is indeed the case !

Now you can try that thought experiment for other kinds of transformations. It's not always easy, but it's rewarding, because the transposition is way more than just a diagonal symmetry on a grid of numbers !