In my math lectures, we talked about the Gram-Determinant where a matrix times its transpose are multiplied together.

Is $A A^\mathrm T$ something special for any matrix $A$?

  • The matrix $A^TA^{-1}$ is generally self similar... – DVD Jun 12 '15 at 3:42
  • 8
    One of the themes of Gilbert Strang's books is the ubiquity of $A^T A$ and $A^T CA$ (with $C$ positive semidefinite) in mathematics. For example, the adjoint of the gradient operator is the negative divergence operator, and the Laplacian is the divergence of the gradient. In one book Strang states, "I have learned to look for $A^T C A$". – littleO Jan 4 '16 at 19:57
  • 1
    Is $AA^T = A^TA$? Would it yield the similar meaning? – A-letubby Jun 1 at 3:18
up vote 130 down vote accepted

The main thing is presumably that $AA^T$ is symmetric. Indeed $(AA^T)^T=(A^T)^TA^T=AA^T$. For symmetric matrices one has the Spectral Theorem which says that we have a basis of eigenvectors and every eigenvalue is real.

Moreover if $A$ is regular, then $AA^T$ is also positive definite, since $$x^TAA^Tx=(A^Tx)^T(A^Tx)> 0$$

Then we have: A matrix is positive definite if and only if it's the Gram matrix of a linear independent set of vectors.

Last but not least if one is interested in how much the linear map respresented by $A$ changes the norm of a vector one can compute

$$\sqrt{\left<Ax,Ax\right>}=\sqrt{\left<A^TAx,x\right>}$$

which simplifies for eigenvectors $x$ to the eigenvalue $\lambda$ to

$$\sqrt{\left<Ax,Ax\right>}=\sqrt \lambda\sqrt{\left<x,x\right>},$$

The determinant is just the product of these eigenvalues.

  • 1
    Then you can write $\mathbb R^n\cong A\bot V$. What is $AA^TA?$ and what is $AA^Tv$ for $v\in V$? How does $AA^T$ hence look like? – Simon Markett Jul 25 '12 at 7:57
  • 6
    What "if $A$ is regular" exactly mean? There seem to be several interpretations on Wikipedia. – qazwsx Mar 11 '14 at 20:21
  • 10
    Regular means here the same as invertible. – Simon Markett Mar 11 '14 at 20:57
  • 1
    What if det($AA^T$) = det($A^TA$). Is there a name for such matrices or any other special thing about them? – Rajendra Uppal Mar 30 '14 at 6:10
  • 1
    @mag, the determinant is multiplicative, so this holds for all square matrices $A$. If $A$ is not square then either side of your equation has to be $0$, as the composition factors through something lower dimensional. – Simon Markett Apr 1 '14 at 14:29

$AA^T$ is positive semi-definite, and in a case in which $A$ is a column matrix, it will be a rank 1 matrix and have only one non-zero eigenvalue which equal to $A^TA$ and its corresponding eigenvector is $A$. The rest of the eigenvectors are the null space of $A$ i.e. $\lambda^TA = 0$.

Indeed, independent of the size of $A$, there is a useful relation in the eigenvectors of $AA^T$ to the eigenvectors of $A^TA$; based on the property that $rank(AA^T)=rank(A^TA)$. That the rank is identical implies that the number of non-zero eigenvectors is identical. Moreover, we can infer the eigenvectors of $A^TA$ from $AA^T$ and vice versa. The eigenvector decomposition of $AA^T$ is given by $AA^Tv_i = \lambda_i v_i$. In case $A$ is not a square matrix and $AA^T$ is too large to efficiently compute the eigenvectors (like it frequently occurs in covariance matrix computation), then it's easier to compute the eigenvectors of $A^TA$ given by $A^TAu_i = \lambda_i u_i$. Pre-multiplying both sides of this equation with $A$ yields

$AA^TAu_i=\lambda_iAu_i$.

Now, the originally searched eigenvectors $v_i$ of $AA^T$ can easily be obtained by $v_i:=Au_i$. Note, that the resulted eigenvectors are not yet normalized.

One could name some properties, like if $B=AA^T$ then

$$B^T=(AA^T)^T=(A^T)^TA^T=AA^T=B,$$

so

$$\langle v,Bw\rangle=\langle Bv,w\rangle=\langle A^Tv,A^Tw\rangle.$$

The product $A^TA$ appears in a key role in the normal equations $A^TAx=A^T b$ for solving linear least squares problems.

  • 12
    What is this key role? – user25959 Apr 16 '16 at 0:07
  • 1
    @lhf, can you extend the properties of this key role? – PlagTag May 6 '17 at 20:54
  • From my guessing its because we get a rectangular matrix --> R with the least squares problem beeing Rx = b. In short: The residuals are orthogonal to the fit line. We aim for finding the minimum squared distance of those. This can be done by an orthogonal projection as we are seeing it here. – PlagTag May 6 '17 at 21:39
  • In the context of linear least squares problems, $A^TA$ is invertible. – cameronroytaylor Mar 8 at 19:22

If you have a real vectorspace equipped with a scalar product, and an Orthogonal matrix $A$ then $AA^T=I$ holds. An Matrix is orthogonal if for the scalarproduct $\langle v,w \rangle = \langle Av, Aw \rangle$ holds for any $v,w \in V$

However I don't see a direkt link to the Gram-Determinant.

Something that occurred to me while reading this answer for help with my homework is that there is a pretty common and important special case, if the linear operator A is normal, i.e. $A^TA=AA^T$. Note this is a stronger condition than saying that $A^TA$ is symmetric, which is always true. In this case, we have that $A$ is diagonalizable. As an obvious special case, $A$ is normal if $A$ is Hermitian (symmeric in the real case).

To see that $A$ normal implies $A^TA$ is diagonalizable, let $\lambda$ be a eigenvalue of $A^TA$ corresponding to the eigenvector x. Then $$(Ax,Ax)=(A^TAx,x)=\lambda(x,x)$$ and similarly $$(A^Tx,A^Tx)=(AA^Tx,x)=\lambda(x,x)$$ where I have used the normality of A. Subtracting these two equations, we have $$((A^T-A)x,(A^T-A)x)=0,$$ which by the definition of inner product implies that $$(A^T-A)x=0 =>A^Tx=Ax.$$ Note that of course this is true whenever $A=A^T$ if A is symmetric, but is a more general condition, since x is an eigenvector, rather than an arbitrary vector. Applying A to both sides of this equation, we have $$A^2x=A^TAx=\lambda x$$.

This shows that if x is an eigenvector of $A^TA$ then it is also an eigenvector of $A^2$, which in turn means it is an eigenvector of $A$ since powers of matrices share the same eigenspace. Therefore, we have constructed a full-rank set of eigenvectors of A, meaning that it is diagonalizable.

To my surprise no one mentioned yet that the root of the Gram determinant of an $n\times k$ matrix $A$ is the $k$-volume of the parallelepiped spanned by the $k$ column vectors of $A$.

  • That totally makes sense since that is enters the integral next to the measure! – Martin Ueding Jul 5 at 13:48

protected by Community Jul 5 '15 at 7:55

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.