1
$\begingroup$

I am doing singular value decomposition on a matrix $R$. The first step is to compute such a matrix $R^TR$. What is this matrix? A reference told me this is cross product of matrix R. I use a programming function and verified it. But what is the relation between this cross product of matrices and cross product of vectors? How can I relate them?

Another reference told me $R^TR$ is called covariance matrix. I checked its definition but I didn't find how can it be equal to the matrix multiplication result of $R^TR$. So is this really covariance matrix? Why?

$\endgroup$
5
  • 1
    $\begingroup$ I don't think there is a name for $R^TR$. $\endgroup$
    – Git Gud
    Apr 6 '14 at 10:26
  • $\begingroup$ What is the meaning of computing $R^TR$? Why doing this? $\endgroup$
    – CyberPlayerOne
    Apr 6 '14 at 10:29
  • 1
    $\begingroup$ Are you asking why the algorithm for computing the SVD decomposition works? $\endgroup$
    – Git Gud
    Apr 6 '14 at 10:29
  • $\begingroup$ Yeah, this is one confusion I have. $\endgroup$
    – CyberPlayerOne
    Apr 6 '14 at 10:35
  • 1
    $\begingroup$ Then you should read a proof of it in a book and come back asking what steps you don't understand, if any. $\endgroup$
    – Git Gud
    Apr 6 '14 at 10:46
1
$\begingroup$

A better term for $R^TR$ would be dot-product matrix, which is what this book chapter calls it, cf. equation (6). The matrix entries of the product $R^T R$ are exactly dot products between columns of $R$. It would be pretty hard to make a connection with the cross-product of vectors in this construction.

The usefulness of $R^T R$ lies in its being symmetric and positive semi-definite for any real matrix $R$, not necessarily square. It can well be called a covariance matrix because of its application in statistics, but the construction also appears in least squares solutions of overdetermined linear systems.

Working with the (real) symmetric matrix $R^T R$ allows us the luxury of having a full basis of eigenvectors, and the corresponding eigenvalues will be nonnegative since $R^T R$ is positive semi-definite as well (indeed, positive definite if $R$ has rank equal to its number of columns).

This amounts to a way of defining the singular values of $R$ as (principal) square roots of the eigenvalues of $R^T R$. Another term for essential the same technique is principal components analysis or PCA, which may be more commonly used in statistics and signal processing.

$\endgroup$
5
  • $\begingroup$ Thanks yes in the book you refered it is called dot-product matrix. But in the R (programming language) $A^TB$ is defined as the matrix crossproduct (stuff.mit.edu/afs/sipb/project/r-project/lib/R/library/base/…). Why the two names are so different? Also, in this SVD tuturial[ling.ohio-state.edu/~kbaker/pubs/…, it says "$AA^T$ is a matrix whose values are the dot product of all the terms, so it is a kind of dispersion matrix of terms throughout all the documents". What does this mean "a kind of dispersion matrix"? $\endgroup$
    – CyberPlayerOne
    Apr 6 '14 at 14:28
  • $\begingroup$ Can the dispersion matrix be defined in many ways? Because in Wikipedia it is defined in this way: en.wikipedia.org/wiki/Covariance_matrix $\endgroup$
    – CyberPlayerOne
    Apr 6 '14 at 14:32
  • $\begingroup$ I think in the book you referred to they call it dot product matrix maybe because $R^TR$ is the dot products of all $R$'s column vectors, and $RR^T$ is the dot products of all $R$'s row vectors. I guess? $\endgroup$
    – CyberPlayerOne
    Apr 6 '14 at 14:37
  • $\begingroup$ They also call it cross-product matrix. See here: utdallas.edu/~herve/abdi-MatrixAlgebra2010-pretty.pdf. Maybe people also call it cross product because it seems $R^TR$ generalizes the cross product from vectors to matrices in the same form. $\endgroup$
    – CyberPlayerOne
    Apr 6 '14 at 14:58
  • 1
    $\begingroup$ Note that the cross-product of two vectors being another vector is peculiar to $\mathbb{R}^3$. I don't doubt that the cross-product construction can be generalized, but if $R^T R$ is such a generalization, I'm not aware of it. $\endgroup$
    – hardmath
    Apr 6 '14 at 15:18
0
$\begingroup$

There is some analogy between constructing $R^T R$ from the matrix $R$ and constructing $\overline{z} z = |z|^2$ from the complex number $z$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.