I am reading the lecture notes of Daniel Spielman on the Conjugate Gradient Descent method (Link to the lecture notes) and he proves therein that the optimal step size for $\alpha$ is given by $$ \frac{2}{\lambda_1 + \lambda_n}.$$ I do not understand how this is derived. In detail: Let $0 < \lambda_1 \leq \ldots \leq \lambda_n$ be the eigenvalues of the psd matrix $A$. Then the stepsize $\alpha$ is given by the minimum w.r.t. $\alpha$ of $$\max_i | 1- \alpha \lambda_i| = |\max(1-\alpha \lambda_1, 1- \alpha \lambda_n) |.$$ I was wondering how I can calculate this solution?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.