3
$\begingroup$

think of an ellipsoid in the n-dimensional space defined by $$(x-\mu)'A(x-\mu)=1.$$

I was calculating the volumes of n-dimensional ellipsoids like the one from above for a while, which is straightforward once the eigenvalues of matrix $A$ are retrieved. The volume $V$ is then given by (using the log scale so that it will not create numerical problems for very large $n$) $$ V_{0} = log(1),\\ V_{1} = log(2), $$ and then for any $i>1$ $$ V_{i} = log(\frac{2\pi}{i-1}) + V_{i-2},$$ and finally $$V = V_{n} + \sum\limits_{i=1}^{n} log(\lambda_{i}^{-\frac{1}{2}}),$$ where each $\lambda_{i}$ refers to one of the $n$ eigenvalues of $A$.

Now we know that each eigenvector of $A$ is the orientation of an axis of the ellipsoid and each $\lambda^{-\frac{1}{2}}$ refers to the half length of one of these axes.

Nevertheless, most (all that I know) mathematical routines that calculate the eigenvalues of $A$ will give the result in a meaningless order. Thus, I will know all half length of the axes of the ellipsoid, but I do not know which eigenvalue corresponds to which eigenvector. I can find this out (I guess) when testing for each eigenvector $v_{k}$ if $$ Av_{k}=\lambda_{i}v_{k}$$ and the $\lambda_{i}$ for which the above holds, will then be the eigenvalue corresponding to $v_{k}$. Nevertheless, my software (R) return eigenvectors normalized to length 1, so the above identity never exists.

My question would be if someone could help me to find out how to sort the eigenvalues anyway so that the first eigenvalue corresponds to the first eigenvector and so on...

Every help is highly appreciated, J

PS: my post is somewhat related to this post and this post, but finally could not figure out how to answer my question with these posts.

$\endgroup$
  • $\begingroup$ All mathematical routines that I know will clearly associate wigenvectors with eigenvalues. Namely sage which I use daily, but I can't ever remember having to find those associations myself. So what kind of mathematical routines are you referring to? $\endgroup$ – MvG Nov 26 '13 at 20:23
  • $\begingroup$ I use the function eigen() in the software R. The eigenvectors are returned in the "correct" order, i.e. when A = covariance( some_data) then the first eigenvector corresponds to the first variable in some_data. The eigenvalues are instead ordered in decreasing order. Makes some sense since the condition number of A will then rely on the first eigenvalue divided by the last one. Nevertheless, this ordering ruins identification of the length of half axes of the ellipsoid and can't be refused in R. $\endgroup$ – Joe1979 Nov 27 '13 at 10:13
3
$\begingroup$

They are associated in R

I'll concentrate on the eigen function from R, as you mention it in a comment. Quoting from its manual:

If ‘r <- eigen(A)’, and ‘V <- r$vectors; lam <- r$values’, then

                         A = V Lmbd V^(-1)                         

(up to numerical fuzz), where Lmbd =‘diag(lam)’.

This implies that the columns of V will match the values in lam in order. The first column of V will always be the eigenvector for the first (i.e. maximal) eigenvalue. lam[i] will always correspond to V[,i].

Eigenvector scale and normalization

You stated:

Nevertheless, my software (R) return eigenvectors normalized to length 1, so the above identity never exists.

But this is not true. Quoting again from the manual:

Recall that the eigenvectors are only defined up to a constant: even when the length is specified they are still only defined up to a scalar of modulus one (the sign for real matrices).

You can verify this easily: if some $v$ satisfies $Av=\lambda v$, then a multiple $w=\mu v$ for some $\mu\neq0$ also satisfies $Aw=\mu Av=\mu\lambda v=\lambda w$ so it, too, will be an eigenvector. This is one of the reasons why it might make more sense to speak about eigenspaces, which are the linear spaces spanned by an eigenvector. The eigenvector isn't uniquely defined by the eigenvalue, but the eigenspace in general is. Choosing a vector of length one is just a choice which is useful in many situations.

Further concerns

Your comment also states that, if A is a covariance matrix, then “the first eigenvector corresponds to the first variable”. I am not sure what you mean by an eigenvector corresponding to a given variable, since these vectors will always have entries for all of the variables. Using some randomized data:

> set.seed(1)
> somedata<-data.frame(x=runif(50), y=runif(50), z=runif(50))
> (A <- cov(somedata))
             x            y            z
x  0.074114067  0.003904344 -0.001744775
y  0.003904344  0.070106650 -0.004266336
z -0.001744775 -0.004266336  0.077143995
> eigen(A)
$values
[1] 0.08086722 0.07362015 0.06687733

$vectors
           [,1]      [,2]       [,3]
[1,] -0.4643667 0.7911076  0.3981360
[2,] -0.4668754 0.1633423 -0.8691068
[3,]  0.7525894 0.5894642 -0.2934980

I see no indication in that eigenvector matrix which supports the claim that the first column were in some way associated with the variable x. So I cannot support the assumption you stated in your comment.

The statement from the manual can indded be verified on this data:

> r <- eigen(A)
> V <- r$vectors; lam <- r$values
> Lmbd <- diag(lam)
> round(V %*% Lmbd %*% t(V) - A, digits=16)
  x y z
x 0 0 0
y 0 0 0
z 0 0 0

If you don't trust that formulation, you may prefer a more explicit eigenvector equation like this:

> round(A %*% V[,1] - lam[1]*V[,1], digits=16)
  [,1]
x    0
y    0
z    0

As a link between these views, you can vectorize the latter for all eigenvectors to obtain a reformulation of the former:

> round(A %*% V - V %*% Lmbd, digits=16)
  [,1] [,2] [,3]
x    0    0    0
y    0    0    0
z    0    0    0

The left hand side of the difference is the matrix A applied to all eigenvectors, whereas the right side is the eigenvectors scaled by their eigenvalues. The differences are zero, so the two are equal. This verifies that the order matches as expected.

$\endgroup$
  • $\begingroup$ First of all: thanks a lot for your explanations. I know understand the relation of eigenvectors to eigenvalues to the meaning of A much better now. And I had to learn that my question was not correct to give me the information I need as an answer. Sorry for that. I would like to edit my question based on the insights I got from your answer, but this would make your answer somewhat inconsistent with the then different question. Could you please advise if I should modify my answer or start a new post? Thx $\endgroup$ – Joe1979 Nov 28 '13 at 15:11
  • $\begingroup$ I'd prefer you create a new post, and edit this question here so that it only contains what you consider relevant here. Maximize the use this question and answer can have for other people, then post a new question based on your new insights. Add linkes between these two questions, either in the text or as comments. And I'll have a look at your next question as well. $\endgroup$ – MvG Nov 28 '13 at 15:54
  • $\begingroup$ I have now created a new post about the mentioned questions here. It would be great if you could help on that as you did so excellently with this (misleading) question. $\endgroup$ – Joe1979 Dec 4 '13 at 11:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.