2
$\begingroup$

To transform the data, below formula is used $[\text{Original Data}]\times[\text{EigenVectors}]=[\text{Transformed Data}]$

Now to recover the original data why cannot we perform $[\text{Original Data}]=[\text{Transformed Data}]\times[\text{EigenVectors}]^{-1}$

when I read most of the material it says to take just the transpose Eigen Vectors matrix, Don't we need to take an Inverse of Eigen Vector Matrix?

additional question below

I went to page and used matrix a. I calculated its contrivance matrix. I got it as below

> b=cov(a,a)
> b
     [,1]   [,2]   [,3]   [,4]
[1,]    0  0.000  0.000  0.000
[2,]    0  0.250 -0.125 -0.125
[3,]    0 -0.125  0.250 -0.125
[4,]    0 -0.125 -0.125  0.250

then eigen vectors and eigen values of the covariance matrix

> eigen(b,TRUE,FALSE)
$values
[1] 3.750000e-01 3.750000e-01 2.775558e-16 0.000000e+00

$vectors
           [,1]       [,2]      [,3] [,4]
[1,]  0.0000000  0.0000000 0.0000000    1
[2,]  0.0000000  0.8164966 0.5773503    0
[3,] -0.7071068 -0.4082483 0.5773503    0
[4,]  0.7071068 -0.4082483 0.5773503    0

all eigen vectors have norm=1, but when I multiplied $vectors matrix by its transpose I didn't get I matrix. I was expecting to get an I matrix becuase as per below comments it seems that inverse = transpose...what did I do wrong?

--------------3 rd modification to question

to check if $vectors matrix's transpose is equal to its inverse, I performed below calculations

> c=eigen(b,TRUE,FALSE)
> d=c$vectors
> d
           [,1]       [,2]      [,3] [,4]
[1,]  0.0000000  0.0000000 0.0000000    1
[2,]  0.0000000  0.8164966 0.5773503    0
[3,] -0.7071068 -0.4082483 0.5773503    0
[4,]  0.7071068 -0.4082483 0.5773503    0
> t(d)#t() function transposes the matrix
     [,1]      [,2]       [,3]       [,4]
[1,]    0 0.0000000 -0.7071068  0.7071068
[2,]    0 0.8164966 -0.4082483 -0.4082483
[3,]    0 0.5773503  0.5773503  0.5773503
[4,]    1 0.0000000  0.0000000  0.0000000
> d*t(d)
          [,1]       [,2]       [,3]      [,4]
[1,] 0.0000000  0.0000000  0.0000000 0.7071068
[2,] 0.0000000  0.6666667 -0.2357023 0.0000000
[3,] 0.0000000 -0.2357023  0.3333333 0.0000000
[4,] 0.7071068  0.0000000  0.0000000 0.0000000

I didn't get d*t(d) = diagonal matrix with diagonal equal to 1...If The transpose of d matrix is equal to the inverse of that matrix, then shouldn't d*t(d) come to diagonal matrix with diagonal equal to 1?

-----------------------4th modification my above calculation for matrix multiplication was wrong. The correct calculation below.

> d %*% t(d)
     [,1]          [,2]          [,3]          [,4]
[1,]    1  0.000000e+00  0.000000e+00  0.000000e+00
[2,]    0  1.000000e+00 -1.665335e-16 -5.551115e-17
[3,]    0 -1.665335e-16  1.000000e+00 -2.220446e-16
[4,]    0 -5.551115e-17 -2.220446e-16  1.000000e+00

--------------5th modification ---------------------------made on 15 Dec

I was looking at Eigen values. I thought that summation of Eigen values will be equal to the summation of variance of each column. Is my thinking wrong? Variances of my data columns are (0,0.25,0.25,0.25) respectively. But my Eigen values dont match up to their summation

$\endgroup$
  • 1
    $\begingroup$ Might it be that in your particular situation the matrix happens to be orthogonal (i.e. its transpose is its inverse)? $\endgroup$ – Henning Makholm Dec 14 '13 at 16:57
  • 1
    $\begingroup$ If the eigenvectors are those of a symmetric matrix whose entries are real, and the eigenvectors are normalized wo that their norm is $1$, then the inverse will be the transpose. $\endgroup$ – Michael Hardy Dec 14 '13 at 17:00
  • $\begingroup$ @MichaelHardy the case that i am talking about is of linear regression - co-variance matrix. My covariance matrix has 6 rows and 6 columns. If the transpose of matrix is equal to inverse of matrix then I thought that [eigen vector matrix]*[eigen vector transposed] will come to 1. But i am not getting 1 :( am i doing any mistake? $\endgroup$ – user2543622 Dec 14 '13 at 17:27
  • $\begingroup$ The covariance matrix is symmetric and its entries are real. Consequently the eigenvalues are real and the eigenvectors are orthogonal to each other. If you normalize the eigenvectors so that their norms are $1$, then the inverse of the matrix whose columns are the eigenvectors will be the transpose of the matrix whose columns are the eigenvectors. Did you transpose the matrix whose columns are the eigenvectors? Or did you transpose the original covariance matrix? You need to work with the right one. $\endgroup$ – Michael Hardy Dec 14 '13 at 17:35
  • $\begingroup$ @MichaelHardy I put my calculations above, what did i do wrong? $\endgroup$ – user2543622 Dec 14 '13 at 20:17
1
$\begingroup$

Look at the matrix you've labeled "$\${}$vectors".

The transpose of that matrix is equal to the inverse of that matrix. With most square matrices that have inverses, the transpose is quite different from the inverse. A square matrix whose transpose is its inverse is called an orthogonal matrix. The finite-dimensional version of what is called the spectral theorem says that that if $M$ is a matrix whose entries are real and $M^\top=M$ (i.e. $M$ is symmetric), then there is an orthogonal matrix $G$ such that $G^\top MG$ is a diagonal matrix. The diagonal entries are the eigevalues. The columns of $G$ are the eigenvectors.

PS: The eigenvalues given in the question are $$ 3.750000e-01 \qquad 3.750000e-01 \qquad 2.775558e-16 \qquad 0.000000e+00$$ So the last two are zero and the first two are $0.375$. The sum of the eigenvalues is therefore the sum of the variances: $0+0.25+0.25+0.25$.

The product of the matrix of eigenvectors and its transpose is reported above to be

> d %*% t(d)
     [,1]          [,2]          [,3]          [,4]
[1,]    1  0.000000e+00  0.000000e+00  0.000000e+00
[2,]    0  1.000000e+00 -1.665335e-16 -5.551115e-17
[3,]    0 -1.665335e-16  1.000000e+00 -2.220446e-16
[4,]    0 -5.551115e-17 -2.220446e-16  1.000000e+00

That is the identity matrix: all of the numbers on the diagonal are $1$ and all off-diagonal entries are $0$. (E.g. $-1.665335e{-}16$ means $-1.665335\times10^{-16}$, so it's $0$, but some rounding was done.)

$\endgroup$
  • $\begingroup$ i transposed matrix 'vectors' and multiplied the transpose by 'vectors'.....i didn't get the I matrix...i have updated my question above...Please let me know what went wrong $\endgroup$ – user2543622 Dec 15 '13 at 0:56
  • $\begingroup$ i added one more sub question, please respond if possible $\endgroup$ – user2543622 Dec 15 '13 at 16:30
  • $\begingroup$ thanks and appreciate your time...and sorry for not seeing numbers properly...i should have realized the issue on my own.. $\endgroup$ – user2543622 Dec 15 '13 at 20:57
1
$\begingroup$

The problem is in your matrix-matrix multiplication:

d*t(d)

In R, matrices are multiplied using %*%, i.e.,

d %*% t(d)

Notice that $d d^T$ cannot have zero on the diagonal unless the corresponding row in $d$ is equal to zero. In your case, none of the diagonal elements in $d d^T$ can be zero.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.