Let's consider smooth and convex minimization problem, i.e. $min f(x)$, where $f$ is not necessarily a quadratic function. If measured by iterations,
- Accelerated Gradient Descend (AGD) has $O(1/T^2)$ rate for weak convex case and linear rate with strong convexity.
- Nonlinear Conjugate Gradient descent (Nonlinear-CGD) has not explicit rate (I got this from Section 5.2 at Numerical Optimization).
But, in quadratic case CGD works much better than APG, i.e. much less iterations are needed to achieve desired precision.
Therefore, is it the same when apply APG and Nonlinear-CGD on above $min f(x)$?