Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to write a program which computes the largest and the smallest (in terms of absolute value) eigenvalues using both Power Iteration and Inverse Iteration.

I can find them using the Inverse Iteration, and I can also find the largest one using the Power Method. But I have no idea how to find the smallest one using the Power Method. I tried applying some kind of shift such as $A - \lambda_{max}I$, but to no avail.

So, how can I modify the Power Method so that it computes the smallest eigenvalue?

Thank you!

share|cite|improve this question
up vote 8 down vote accepted

If you know that $A$ is symmetric positive-definite, then the spectral shift $B = A-\lambda_\max I$ will work. Use the power method on $B$, then add $\lambda_\max$ to the result to get the smallest eigenvalue of $A$.

The reason this shift works is that a positive-definite matrix has all positive eigenvalues. Therefore $B$ has all non-positive eigenvalues, with the smallest eigenvalue of $A$ now the largest-magnitude (most negative) eigenvalue of $B$. The power method will then find that eigenvalue.

The same approach works for negative-definite matrices, for the same reason.

share|cite|improve this answer
    
This seems to work for symmetric negative-definite matrices as well. – ivt Jan 7 '13 at 1:55
    
Yes. I'll make a note of that. – user7530 Jan 7 '13 at 1:56
    
While this is the best result we've got so far, any thoughts on symmetric indefinite matrices are still very welcome! – ivt Jan 7 '13 at 2:02
1  
Well, using this approach you can always find the largest (most positive) and smallest (most negative) eigenvalues. But for indefinite matrices it won't help you find the least-magnitude eigenvalue. – user7530 Jan 7 '13 at 2:05
    
This is true, but it is one thing if you are doing this for fun or homework, but in practice computing the smallest eigenvalue with the power method can be very very slow (I believe it is linear). – csta Apr 30 '14 at 12:46

Suppose $A$ is invertible and has eigenvalue $\lambda$. Then $A^{-1}$ has eigenvalue $\lambda^{-1}$: this follows straight from the eigenvector equation $$Av = \lambda v \Rightarrow \frac{v}{\lambda} = A^{-1}v.$$

Since the smallest eigenvalue of $A$ is the largest eigenvalue of $A^{-1}$, you can find it using power iteration on $A^{-1}$:

$$v_{i+1} = A^{-1} \frac{v_i}{\|v_i\|}.$$

Unfortunately you now have to perform a linear solve at each iteration (or compute a decomposition of $A$), instead of just taking matrix-vector products.

share|cite|improve this answer
1  
Thank you! This is exactly the Inverse Iteration and I already know how to use it. The trick is to find the smallest one using the "pure" Power Method, without inverting $A$ or linear solve at each step. – ivt Jan 7 '13 at 1:21
    
I'm afraid I'm not aware of any "purer" way of getting at the smallest eigenvalue. – user7530 Jan 7 '13 at 1:32
    
I'm reading that using the shift $\sigma = \lambda_{max}$ indeed works, but only for positive-definite matrices. Can someone confirm this? – ivt Jan 7 '13 at 1:39
    
Yes. I'll write up a second answer. – user7530 Jan 7 '13 at 1:41

A general creative answer to this question can be composed the following way:

1)Your symmetric/hermitian Matrix $H$ has a spectrum with positive and negative eigenvalues. Assume you can calculate the eigenvalue with maximum absolute value $\omega$ using power method.

2)Shift the matrix by a constant $\lambda$ to target the part of the spectrum that you are interested $H-\lambda I$. In case you are interested in the smallest magnitude eigenvalue $\lambda = 0$.

3)Square your matrix $H'= (H-\lambda I)^2$. This will make your matrix positive definite.

4)Now the desired eigenvalue will be as close to zero, while the change in the largest magnitude eigenvalue can be computed trivially. You can also use the fact that the matrix spectrum will be bounded by the Hilbert-Schmidt norm and avoid step 1.

5)Shift by the maximum eigenvalue/bound $H'=H'- max(spec(H'))I$. Now you will have a negative-definite matrix with the targeted eigenvalue $x'$ having the highest magnitude which you can compute using power-method.

6)Solve for the variable $x$: $x'=(x-\lambda)^2+max(spec(H'))$ You don't have the information of whether your initial eigenvalue was positive or negative but this can be found by multiplying the original matrix to the eigenvector.

In case you have a Hermitian matrix you have to use the inner product defined for complex Hilbert spaces.

share|cite|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.