When performing QZ iterations on matrices A and B where A is upper hessenberg to begin with and B is upper triangular, how can one tell when the algorithm has converged? I understand that if it consists of only real eigenvalues then the subdiagonal entries of A will approach zero. However, complex eigenvalues remain below the diagonal of A. Is there a way to determine when all of the values below the subdiagonal of A that are supposed to be zero are in fact zero while the ones that are supposed to remain are still there?
The QR and QZ algorithm split the task into subtasks on corresponding upper and lower diagonal blocks whenever one of the subdiagonal entries is deemed to be equal to zero. This recursion stops whenever the block size is 1 or 2.
In blocksize 2, the eigenvalues corresponding to these blocks can be computed by solving the quadratic characteristic equation, typically they should be complex conjugate pairs, but also a real pair can happen. In the latter case, the necessary rotation to diagonal form can be computed from the eigenvalues. And also in the complex case one might rotate the block into the standard matrix representation of a complex number.
See http://www.netlib.org/lapack/lawnspdf/lawn173.pdf on the QZ algorithm using implicit multishift and chasing chains of small bulge-pairs. Discussion of deflation and infinite eigenvalues starts on p.9-10.