2
$\begingroup$

I am reading the lecture notes of Daniel Spielman on the Conjugate Gradient Descent method (Link to the lecture notes) and he proves therein that the optimal step size for $\alpha$ is given by $$ \frac{2}{\lambda_1 + \lambda_n}.$$ I do not understand how this is derived. In detail: Let $0 < \lambda_1 \leq \ldots \leq \lambda_n$ be the eigenvalues of the psd matrix $A$. Then the stepsize $\alpha$ is given by the minimum w.r.t. $\alpha$ of $$\max_i | 1- \alpha \lambda_i| = |\max(1-\alpha \lambda_1, 1- \alpha \lambda_n) |.$$ I was wondering how I can calculate this solution?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.