1
$\begingroup$

I am doing singular value decomposition on a matrix $R$. The first step is to compute such a matrix $R^TR$. What is this matrix? A reference told me this is cross product of matrix R. I use a programming function and verified it. But what is the relation between this cross product of matrices and cross product of vectors? How can I relate them?

Another reference told me $R^TR$ is called covariance matrix. I checked its definition but I didn't find how can it be equal to the matrix multiplication result of $R^TR$. So is this really covariance matrix? Why?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ I don't think there is a name for $R^TR$. $\endgroup$ – Git Gud Apr 6 '14 at 10:26
  • $\begingroup$ What is the meaning of computing $R^TR$? Why doing this? $\endgroup$ – Tyler 十三将士归玉门 Apr 6 '14 at 10:29
  • 1
    $\begingroup$ Are you asking why the algorithm for computing the SVD decomposition works? $\endgroup$ – Git Gud Apr 6 '14 at 10:29
  • $\begingroup$ Yeah, this is one confusion I have. $\endgroup$ – Tyler 十三将士归玉门 Apr 6 '14 at 10:35
  • 1
    $\begingroup$ Then you should read a proof of it in a book and come back asking what steps you don't understand, if any. $\endgroup$ – Git Gud Apr 6 '14 at 10:46
1
$\begingroup$

A better term for $R^TR$ would be dot-product matrix, which is what this book chapter calls it, cf. equation (6). The matrix entries of the product $R^T R$ are exactly dot products between columns of $R$. It would be pretty hard to make a connection with the cross-product of vectors in this construction.

The usefulness of $R^T R$ lies in its being symmetric and positive semi-definite for any real matrix $R$, not necessarily square. It can well be called a covariance matrix because of its application in statistics, but the construction also appears in least squares solutions of overdetermined linear systems.

Working with the (real) symmetric matrix $R^T R$ allows us the luxury of having a full basis of eigenvectors, and the corresponding eigenvalues will be nonnegative since $R^T R$ is positive semi-definite as well (indeed, positive definite if $R$ has rank equal to its number of columns).

This amounts to a way of defining the singular values of $R$ as (principal) square roots of the eigenvalues of $R^T R$. Another term for essential the same technique is principal components analysis or PCA, which may be more commonly used in statistics and signal processing.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks yes in the book you refered it is called dot-product matrix. But in the R (programming language) $A^TB$ is defined as the matrix crossproduct (stuff.mit.edu/afs/sipb/project/r-project/lib/R/library/base/…). Why the two names are so different? Also, in this SVD tuturial[ling.ohio-state.edu/~kbaker/pubs/…, it says "$AA^T$ is a matrix whose values are the dot product of all the terms, so it is a kind of dispersion matrix of terms throughout all the documents". What does this mean "a kind of dispersion matrix"? $\endgroup$ – Tyler 十三将士归玉门 Apr 6 '14 at 14:28
  • $\begingroup$ Can the dispersion matrix be defined in many ways? Because in Wikipedia it is defined in this way: en.wikipedia.org/wiki/Covariance_matrix $\endgroup$ – Tyler 十三将士归玉门 Apr 6 '14 at 14:32
  • $\begingroup$ I think in the book you referred to they call it dot product matrix maybe because $R^TR$ is the dot products of all $R$'s column vectors, and $RR^T$ is the dot products of all $R$'s row vectors. I guess? $\endgroup$ – Tyler 十三将士归玉门 Apr 6 '14 at 14:37
  • $\begingroup$ They also call it cross-product matrix. See here: utdallas.edu/~herve/abdi-MatrixAlgebra2010-pretty.pdf. Maybe people also call it cross product because it seems $R^TR$ generalizes the cross product from vectors to matrices in the same form. $\endgroup$ – Tyler 十三将士归玉门 Apr 6 '14 at 14:58
  • 1
    $\begingroup$ Note that the cross-product of two vectors being another vector is peculiar to $\mathbb{R}^3$. I don't doubt that the cross-product construction can be generalized, but if $R^T R$ is such a generalization, I'm not aware of it. $\endgroup$ – hardmath Apr 6 '14 at 15:18
0
$\begingroup$

There is some analogy between constructing $R^T R$ from the matrix $R$ and constructing $\overline{z} z = |z|^2$ from the complex number $z$.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.