# Simultaneous Iteration, Convergence to Eigenvectors

I have a question about the simultaneous iteration. I am currently working for an exam and I do not understand this step (taken from Numerical Linear Algebra from Trefethen/Bau):

For the power iteration it holds, that for an arbitrary starting vector $v^{(0)}$ with $||v^{(0)}|| = 1$ that $A^kv \rightarrow q_j$ for $k \rightarrow \infty$, where $q_j$ is the eigenvector corresponding to the maximum eigenvalue (in absolute value). So far, so good.

Now the book states the following: For a set of n linearly independent vectors $v_1^{(0)}, ..., v_n^{(0)}$, the space $span\{A^kv_1^{(0)}, ..., A^kv_n^{(0)}\}$ converges to the space $span\{q_1, ..., q_n\}$, i.e. the space spanned by the eigenvectors. And this is something I do not understand. Above it is stated, that for almost any arbitrary starting vector, $v$ would converge to the eigenvector corresponding to the largest eigenvalue. How can than the space spanned by several vectors span the same space as all eigenvectors? I would assume that the all will converge sooner or later to the same vector, thus only spanning a one-dimensional subspace.

Where is my mistake?

-
$A^kv$ doesn't converge to $q_j$ unless the corresponding eigenvalue is $1$; you need to normalize. – joriki Mar 2 '12 at 22:36

The problem is that, for suitable matrices and generic $n$-sets of vectors, $\text{Span} \{A^k v_1, \cdots, A^k v_n\}$ is a $n$-dimension vector space. And a sequence of $n$-dimensional vector spaces won't converge in any reasonable sense to a $1$-dimensional vector space, for $n>1$. In the limit, the vectors $A^k v_1$, ... , $A^k v_n$ each converge (at least in the projective space) to a single vector. But, for as long as you iterate the matrix $A$, these vectors won't be exactly the same, and this small difference is what makes their spanned linear space large. For instance, if we take:

$$A := \left( \matrix{1 & 0 & 0 \\ 0 & 1/2 & 0 \\ 0 & 0 & 1/4}\right),$$

$$v_1 := (1,1,1), \ v_2 := (1,2,3),$$

then:

$$A^k v_1 := (1,2^{-k}, 4^{-k}), \ A^k v_2 := (1, 2 \cdot2^{-k}, 3 \cdot 4^{-k}),$$

and you can check that those two vectors still generate a two-dimensional vector space. Moreover, as $k$ goes to $+ \infty$, this two-dimensional vector space will converge to the space $\{(x,y,0): \ x,y \in \mathbb{R}\}$, which is the vector subspace spanned by the eigenvectors $(1,0,0)$ and $(0,1,0)$ corresponding to the two eigenvalues of maximal modulus.

-