# Transpose of a matrix and the product $A A^\top$

I've been following 3Blue1Brown's Essence of Linear Algebra, basically the question (1) is what is the geometric meaning of the transpose? I've watched Chapter 9-Dot Products and duality . I can see that the transpose of a matrix has something to do with duality and dual spaces, but I'm not able to put my finger on it.

There is an answer, 3B1B Transpose, in this answer there is a sentence,

When you transpose a matrix, you are actually making use of this vector-dual vector identification to change your transformation to act on the dual vectors instead of the original vectors.

I've been reading this sentence again and again, but I'm having a hard time understanding the visual picture behind this, can someone please provide an example with a 2 $$\times$$ 2 matrix? What's the exact relation between the following two interetations,

1)$$\begin{bmatrix} a & b \\ c & d \end{bmatrix}\begin{bmatrix} x\\ y \end{bmatrix}=x\begin{bmatrix} a\\ c \end{bmatrix}+y\begin{bmatrix} b\\ d \end{bmatrix}$$

2)

I started Gilbert Strang's, Linear Algebra course, and there's a lecture about projection matrices, and solving systems that have no solutions. He basically uses the product $$A A^T$$ instead of just $$A$$.

question(2): What's the geometric meaning of $$A A^T$$

Regarding the transpose: First, a caveat: in practice, the transpose is rarely thought of in a "geometric" way. Instead, we think about it in terms of its defining property, which is to say that for all vectors $$x$$ and $$y$$ of the correct size, the dot-product $$(Ax)^Ty$$ is the same as $$x^T(A^Ty)$$. Equivalently, if we convert the mapping $$x^T \mapsto x^TA$$ on row-vectors into a map on column vectors, we end up with the map $$x \mapsto Ax$$. More abstractly, we might think of the transpose in terms of its relation to the adjoint on the dual space associated with the linear map $$T: V \to W$$. In other words, intuition for the transpose tends not to come from "what the transformation looks like", but from "how the transformation fits together with other things (i.e. vectors, covectors, and other transformations)".

That being said: if you're looking for a geometric idea of what the transpose does, the best way to get that is through the polar decomposition (note: the article linked deals with matrices with complex entries, but I will focus on matrices with real entries). Every transformation can be written in the form $$A = PU$$, where $$P$$ is a positive semidefinite matrix, and $$U$$ is an orthogonal matrix. That is, $$A$$ can be broken down into a rotation/reflection (encoded by $$U$$), followed by a stretch/squish along perpendicular axes (encoded by $$P$$). The transpose is given by $$A^T = (PU)^T = U^TP^T = U^{-1}P.$$ That is, the transformation that is done by the transposed matrix is the same stretch along perpendicular axes, followed by the reverse of the rotation/reflection $$U$$.

Regarding $$AA^T$$: As it turns out, the matrix $$P$$ from the polar decomposition $$A = PU$$ discussed above is given by $$P = \sqrt{AA^T}$$. Moreover, $$AA^T$$ is itself a positive semidefinite matrix. What we might say for the geometric idea behind $$AA^T$$, then, is that What we get out of all this is that $$A^TA$$ encodes all of the streching/squishing that $$A$$ does.

• A lot of what I have said is taken from my old answer here, which you might find interesting. I am surprised, given that you are looking at Strang's linear algebra course, that you are thinking about the matrix $AA^T$ rather than $A^TA$. Notably, $A^TA$ comes up whenever least-squares approximate answers are involved. Jan 19 '20 at 12:24
• As for your question "what's the exact relation between the following two interetations...", I really don't see how it connects to the discussion of the transpose of a matrix. I would suggest that it is better asked as a separate question Jan 19 '20 at 12:28
• If you're still interested in seeing an example, I could add one tomorrow. In my office I have matlab, which I could use to generate a graphical aide. Jan 19 '20 at 12:30
• "I am surprised, given that you are looking at Strang's linear algebra course, that you are thinking about the matrix $AA^T$ rather than $A^TA$" Yeah, that was my mistake, I intended to type $A^TA$ So if I were to animate transpose, it would be best to look at it from a polar decomp, POV. Thanks a lot, I haven't completely understood the exact idea though. Shall I revert back after some thought?\ Jan 21 '20 at 12:33
• "If you're still interested in seeing an example, I could add one tomorrow. In my office I have matlab, which I could use to generate a graphical aide" Yes of course, very sorry for the late response. I had some trouble connecting to the internet. Jan 21 '20 at 12:33