309
$\begingroup$

In my math lectures, we talked about the Gram-Determinant where a matrix times its transpose are multiplied together.

Is $A A^\mathrm T$ something special for any matrix $A$?

$\endgroup$
5
  • $\begingroup$ The matrix $A^TA^{-1}$ is generally self similar... $\endgroup$
    – DVD
    Jun 12, 2015 at 3:42
  • 20
    $\begingroup$ One of the themes of Gilbert Strang's books is the ubiquity of $A^T A$ and $A^T CA$ (with $C$ positive semidefinite) in mathematics. For example, the adjoint of the gradient operator is the negative divergence operator, and the Laplacian is the divergence of the gradient. In one book Strang states, "I have learned to look for $A^T C A$". $\endgroup$
    – littleO
    Jan 4, 2016 at 19:57
  • 1
    $\begingroup$ Is $AA^T = A^TA$? Would it yield the similar meaning? $\endgroup$
    – A-letubby
    Jun 1, 2018 at 3:18
  • 1
    $\begingroup$ It's symmetric and I believe positive. $\endgroup$
    – user193319
    Apr 11, 2019 at 14:14
  • 6
    $\begingroup$ @A-letubby No, $AA^T$ is not necessarily $=A^TA$ $\endgroup$
    – lmaooooo
    Mar 19, 2020 at 13:50

9 Answers 9

Reset to default
208
$\begingroup$

The main thing is presumably that $AA^T$ is symmetric. Indeed $(AA^T)^T=(A^T)^TA^T=AA^T$. For symmetric matrices one has the Spectral Theorem which says that we have a basis of eigenvectors and every eigenvalue is real.

Moreover if $A$ is invertible, then $AA^T$ is also positive definite, since $$x^TAA^Tx=(A^Tx)^T(A^Tx)> 0$$

Then we have: A matrix is positive definite if and only if it's the Gram matrix of a linear independent set of vectors.

Last but not least if one is interested in how much the linear map represented by $A$ changes the norm of a vector one can compute

$$\sqrt{\left<Ax,Ax\right>}=\sqrt{\left<A^TAx,x\right>}$$

which simplifies for eigenvectors $x$ to the eigenvalue $\lambda$ to

$$\sqrt{\left<Ax,Ax\right>}=\sqrt \lambda\sqrt{\left<x,x\right>},$$

The determinant is just the product of these eigenvalues.

$\endgroup$
12
  • 1
    $\begingroup$ Then you can write $\mathbb R^n\cong A\bot V$. What is $AA^TA?$ and what is $AA^Tv$ for $v\in V$? How does $AA^T$ hence look like? $\endgroup$
    – Simon Markett
    Jul 25, 2012 at 7:57
  • 7
    $\begingroup$ What "if $A$ is regular" exactly mean? There seem to be several interpretations on Wikipedia. $\endgroup$
    – qazwsx
    Mar 11, 2014 at 20:21
  • 12
    $\begingroup$ Regular means here the same as invertible. $\endgroup$
    – Simon Markett
    Mar 11, 2014 at 20:57
  • 1
    $\begingroup$ What if det($AA^T$) = det($A^TA$). Is there a name for such matrices or any other special thing about them? $\endgroup$
    – Rajendra Uppal
    Mar 30, 2014 at 6:10
  • 1
    $\begingroup$ @mag, the determinant is multiplicative, so this holds for all square matrices $A$. If $A$ is not square then either side of your equation has to be $0$, as the composition factors through something lower dimensional. $\endgroup$
    – Simon Markett
    Apr 1, 2014 at 14:29
31
$\begingroup$

$AA^T$ is positive semi-definite, and in a case in which $A$ is a column matrix, it will be a rank 1 matrix and have only one non-zero eigenvalue which equal to $A^TA$ and its corresponding eigenvector is $A$. The rest of the eigenvectors are the null space of $A$ i.e. $\lambda^TA = 0$.

Indeed, independent of the size of $A$, there is a useful relation in the eigenvectors of $AA^T$ to the eigenvectors of $A^TA$; based on the property that $rank(AA^T)=rank(A^TA)$. That the rank is identical implies that the number of non-zero eigenvectors is identical. Moreover, we can infer the eigenvectors of $A^TA$ from $AA^T$ and vice versa. The eigenvector decomposition of $AA^T$ is given by $AA^Tv_i = \lambda_i v_i$. In case $A$ is not a square matrix and $AA^T$ is too large to efficiently compute the eigenvectors (like it frequently occurs in covariance matrix computation), then it's easier to compute the eigenvectors of $A^TA$ given by $A^TAu_i = \lambda_i u_i$. Pre-multiplying both sides of this equation with $A$ yields

$AA^TAu_i=\lambda_iAu_i$.

Now, the originally searched eigenvectors $v_i$ of $AA^T$ can easily be obtained by $v_i:=Au_i$. Note, that the resulted eigenvectors are not yet normalized.

$\endgroup$
18
$\begingroup$

Special? Yes! The matrix $A^TA$ is abundant with the Least Squares Finite Element Method in Numerical Analysis:

EDIT. Another interesting application of the specialty of $A^TA$ is perhaps the following. Suppose that we have a dedicated matrix inversion routine at our disposal, namely for a matrix $A$ with pivots only at the main diagonal. How can we use this routine for inverting an arbitrary matrix $A$ ? Assuming that the inverse $A^{-1}$ does exist and nothing else is known. As follows.
  1. Multiply $A$ on the left with $A^T$, giving $B = A^T A$ .
  2. The inverse can of $B$ can be determined by employing our special matrix inversion routine.
    The reason is that the pivots of $B$ are always at the main diagonal: see the first reference.
  3. The inverse matrix is $B^{-1} = (A^TA)^{-1} = A^{-1}A^{-T}$ .
    Therefore multiply on the right with $A^T$ , giving : $A^{-1}A^{-T}A^T = A^{-1}$ .
    The inverse that has been sought for is recovered herewith.
Software demo (Delphi Pascal): 
program Demo;

type
matrix = array of array of double;


procedure Inverteren(var b : matrix);
{
Matrix inversion
pivots on diagonal
}
var
pe : double;
NDM,i,j,k : integer;
begin
NDM := Length(b);
for k := 0 to NDM-1 do
begin
pe := b[k,k];
b[k,k] := 1;
for i := 0 to NDM-1 do
begin
b[i,k] := b[i,k]/pe;
if (i = k) then Continue;
for j := 0 to NDM-1 do
begin
if (j = k) then Continue;
b[i,j] := b[i,j] - b[i,k]*b[k,j];
end;
end;
for j := 0 to NDM-1 do
b[k,j] := - b[k,j]/pe;
b[k,k] := 1/pe;
end;
end;


procedure inverse(b : matrix; var q : matrix);
{
Matrix inversion
without pivoting
}
var
NDM,i,j,k : integer;
p : matrix;
h : double;
begin
NDM := Length(b);
SetLength(p,NDM,NDM);
{ Transpose * original }
for i := 0 to NDM-1 do
begin
for j := 0 to NDM-1 do
begin
h := 0;
for k := 0 TO NDM-1 do
h := h + b[k,i]b[k,j];
p[i,j] := h;
end;
end;
Inverteren(p);
SetLength(q,NDM,NDM);
{ (B^TB)^(-1)*B^T = B^(-1) }
for i := 0 to NDM-1 do
begin
for j := 0 to NDM-1 do
begin
h := 0;
for k := 0 TO NDM-1 do
h := h + p[i,k]*b[j,k];
q[i,j] := h;
end;
end;
end;


procedure test(NDM : integer);
var
i,j,k : integer;
b,q : matrix;
h : double;
begin
SetLength(b,NDM,NDM);
Random; Random;
for i := 0 to NDM-1 do
begin
for j := 0 to NDM-1 do
begin
b[i,j] := Random;
end;
end;
inverse(b,q);
for i := 0 to NDM-1 do
begin
for j := 0 to NDM-1 do
begin
h := 0;
for k := 0 TO NDM-1 do
h := h + q[i,k]*b[k,j];
Write(h:5:2,' ');
end;
Writeln;
end;
end;


BEGIN
test(9);
END.
Output:

 1.00  0.00  0.00 -0.00  0.00  0.00  0.00  0.00 -0.00 
 0.00  1.00  0.00 -0.00  0.00  0.00  0.00  0.00 -0.00 
-0.00  0.00  1.00  0.00 -0.00 -0.00 -0.00 -0.00  0.00 
-0.00 -0.00 -0.00  1.00 -0.00 -0.00 -0.00 -0.00  0.00 
 0.00  0.00  0.00  0.00  1.00  0.00  0.00  0.00  0.00 
 0.00  0.00  0.00 -0.00 -0.00  1.00  0.00  0.00 -0.00 
-0.00 -0.00 -0.00 -0.00 -0.00 -0.00  1.00 -0.00  0.00 
-0.00 -0.00 -0.00  0.00 -0.00 -0.00 -0.00  1.00  0.00 
-0.00 -0.00 -0.00 -0.00 -0.00 -0.00 -0.00 -0.00  1.00

LATE EDIT.
Theory of Polar Decomposition is described in Wikipedia. If attention is restricted to real-valued (non-singular square invertible) matrices, then an appropriate question and some answers are found in Polar decomposition of real matrices. Especially the following formula over there leaves no doubt that a matrix multiplied with its transpose IS something special: $$ B = \left[B\left(B^TB\right)^{-1/2}\right]\left[\left(B^TB\right)^{1/2}\right] $$ Least Squares methods (employing a matrix multiplied with its transpose) are also very useful with Automated Balancing of Chemical Equations

$\endgroup$
10
$\begingroup$

One could name some properties, like if $B=AA^T$ then

$$B^T=(AA^T)^T=(A^T)^TA^T=AA^T=B,$$

so

$$\langle v,Bw\rangle=\langle Bv,w\rangle=\langle A^Tv,A^Tw\rangle.$$

$\endgroup$
6
$\begingroup$

Something that occurred to me while reading this answer for help with my homework is that there is a pretty common and important special case, if the linear operator A is normal, i.e. $A^TA=AA^T$. Note this is a stronger condition than saying that $A^TA$ is symmetric, which is always true. In this case, we have that $A$ is diagonalizable. As an obvious special case, $A$ is normal if $A$ is Hermitian (symmeric in the real case).

To see that $A$ normal implies $A^TA$ is diagonalizable, let $\lambda$ be a eigenvalue of $A^TA$ corresponding to the eigenvector x. Then $$(Ax,Ax)=(A^TAx,x)=\lambda(x,x)$$ and similarly $$(A^Tx,A^Tx)=(AA^Tx,x)=\lambda(x,x)$$ where I have used the normality of A. Subtracting these two equations, we have $$((A^T-A)x,(A^T-A)x)=0,$$ which by the definition of inner product implies that $$(A^T-A)x=0 =>A^Tx=Ax.$$ Note that of course this is true whenever $A=A^T$ if A is symmetric, but is a more general condition, since x is an eigenvector, rather than an arbitrary vector. Applying A to both sides of this equation, we have $$A^2x=A^TAx=\lambda x$$.

This shows that if x is an eigenvector of $A^TA$ then it is also an eigenvector of $A^2$, which in turn means it is an eigenvector of $A$ since powers of matrices share the same eigenspace. Therefore, we have constructed a full-rank set of eigenvectors of A, meaning that it is diagonalizable.

$\endgroup$
5
$\begingroup$

To my surprise no one mentioned yet that the root of the Gram determinant of an $n\times k$ matrix $A$ is the $k$-volume of the parallelepiped spanned by the $k$ column vectors of $A$.

$\endgroup$
1
  • $\begingroup$ That totally makes sense since that is enters the integral next to the measure! $\endgroup$
    – Martin Ueding
    Jul 5, 2018 at 13:48
4
$\begingroup$

The product $A^TA$ appears in a key role in the normal equations $A^TAx=A^T b$ for solving linear least squares problems.

$\endgroup$
4
  • 24
    $\begingroup$ What is this key role? $\endgroup$
    – Matthew C
    Apr 16, 2016 at 0:07
  • 3
    $\begingroup$ @lhf, can you extend the properties of this key role? $\endgroup$
    – PlagTag
    May 6, 2017 at 20:54
  • $\begingroup$ From my guessing its because we get a rectangular matrix --> R with the least squares problem beeing Rx = b. In short: The residuals are orthogonal to the fit line. We aim for finding the minimum squared distance of those. This can be done by an orthogonal projection as we are seeing it here. $\endgroup$
    – PlagTag
    May 6, 2017 at 21:39
  • 1
    $\begingroup$ In the context of linear least squares problems, $A^TA$ is invertible. $\endgroup$
    – cameronroytaylor
    Mar 8, 2018 at 19:22
4
$\begingroup$

If you have a real vector space equipped with a scalar product, and an Orthogonal matrix $A$ then $AA^T=I$ holds. A matrix is orthogonal if for the scalar product $\langle v,w \rangle = \langle Av, Aw \rangle$ holds for any $v,w \in V$

However I don't see a direct link to the Gram-Determinant.

$\endgroup$
0
$\begingroup$

The matrix $A$ represents a rotation if $A^TA=I$ and Det A=1 (where $A^T$ means $A$ transpose)

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .