# Properties of the A-transpose-A matrix

I believe that $$A^TA$$ is a key matrix structure because of its connection to variance-covariance matrices.

In Professor Strang's linear algebra lectures, "A-transpose-A" - with this nomenclature, as opposed to $$X'X$$, for example - is the revolving axis.

Yet, it is not easy to find on a quick Google search a list of its properties. I presume that part of the reason may be that they are shared by variance-covariance matrices. But I'd like to confirm this (does it have identical properties to a var-cov matrix?), and have the list easily available from now on here at SE-Mathematics.

Just to not shy away from the initial effort, here is what I think I have so far:

1. Symmetry
2. Positive semidefinite-ness
3. Real and positive eigenvalues
4. The trace is positive (the trace is the sum of eigenvalues)
5. The determinant is positive (the determinant is the product of the eigenvalues)
6. The diagonal entries are all positive
7. Orthogonal eigenvectors (**)
8. Diagonalizable as $$Q\Lambda Q^T$$
9. It is possible to obtain a Cholesky decomposition.
10. Rank of $$A^TA$$ is the same as rank of $$A$$.
11. $$\text{ker}(A^TA)=\text{ker}(A)$$

(**) The eigenvectors of A-transpose-A form the matrix $$V$$ in singular value decomposition (SVD) of $$A,$$ while the square root of the eigenvalues of A-transpose-A are the singular values of the SVD. Similarly, the eigenvectors of A-A-transpose $$AA^\top$$ include the columns in the matrix $$U$$ of the SVD of $$A.$$ The importance of this is exemplified in the fact that SVD can be used to solve least squares regression by computing the Penrose-Moore pseudo-inverse $$A^\dagger = V\Sigma^\dagger U^*,$$ although the QR decomposition is a more expedient computational method.

There is a nice post on the topic here.

• Item 1 and 2 are straight forward. Item is a result due to symmetry. It is usually discussed before or in context of eigenvalue decomposition or Jordan normal form Commented Aug 18, 2016 at 23:22
• It is a key matrix structure because of the role it plays in orthogonal projection. Covariance matrices are just special cases. Commented Aug 18, 2016 at 23:51
• This is known as a Gramian matrix en.wikipedia.org/wiki/Gramian_matrix Commented Aug 19, 2016 at 0:44
• Equivalently, you are asking for a list of properties of symmetric positive semidefinite matrices (because every s.p.s.d. matrix can be written as $A^TA$ for some $A$). A bunch of such properties are listed on the Wikipedia page.
– user856
Commented Aug 19, 2016 at 1:33
• @symplectomorphic I have been intrigued by your comment for a while now, and would like to get some reference summarizing this issue. For instance, it is immediate to think about ordinary linear regression and PCA, both being forms of projection (over column space of the model matrix, or on the eigenvectors of the covariance). However, I am shooting for something more general, more conceptual... Commented Sep 7, 2016 at 21:25

Yes, it has all the properties of a covariance matrix because it is one. You can define a multivariate normal distribution for which $A^T A$ is the covariance matrix.
• You can't literally list all the properties, because there are infinitely many. Of course, many of those are trivial consequences of other properties, others so obscure that nobody would ever care about them. But everything follows pretty much from the orthogonal diagonalization: $A^T A = U^T D U$ where $U$ is a real orthogonal matrix and $D$ is diagonal with nonnegative diagonal elements. Commented Aug 18, 2016 at 23:35
• If $Z$ is a vector whose entries are iid standard normal random variables, $A^T Z$ has mean $0$ and covariance matrix $A^T A$. Commented Oct 2, 2023 at 1:30