Let us assume that we want to solve the linear system:

$$\mathbf{A}\mathbf{x} = \mathbf{b}$$

with the conjugate gradient method. $\mathbf{A}$ is a positive semi-definite Hermitian matrix.

The computational complexity of the conjugate gradient method is given as $O(\sqrt{\kappa} m)$, where $m$ is the number of non-zero entries in $\mathbf{A}$ and $\kappa = \frac{\lambda_{\text{max}}}{\lambda_{\text{min}}}$ (the condition number of $\mathbf{A}$). When the matrix is positive semi-definite the smallest eigenvalue is $0$. And yet the conjugate gradient method converges (and can even converge very rapidly). What am I missing here? Is the value of $\lambda_{\text{min}}$ supposed to be the smallest non-zero eigenvalue or does the computational complexity not hold for a positive semi-definite Hermitian matrix?

$$\boldsymbol{b} \in R(\boldsymbol{A})$$

  • $\begingroup$ If $A$ is PSD, then the linear system $A x = b$ has infinitely many solutions. You find one of them, not the one solution. $\endgroup$ – Rodrigo de Azevedo May 30 '16 at 19:03
  • $\begingroup$ Thanks, you are right, but I am trying to ascertain the convergence properties of CG, it finds a possible solution to the problem, but what bounds its convergence complexity. $\endgroup$ – trienko Jun 1 '16 at 6:42
  • $\begingroup$ Try using the smallest (in absolute value) nonzero eigenvalue to compute the modified condition number. $\endgroup$ – Rodrigo de Azevedo Jun 1 '16 at 12:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.