I'm trying to understand a proof of the descent lemma, which says that if f is a continuously differentiable function over $\mathbb{R}^n$ with L-Lipschitz continuous gradient. Then, for any $x, y \in \mathbb{R}^n$:

$f(y) \leq f(x) + \nabla f(x)^T(y-x)+ \frac{L}{2}||x-y||^2$

By using the Fundamental Theorem of Calculus:

$\int_a^b f(x) dx = F(b) - F(a) \qquad F' = f$

And by defining:

$ g(t) = f(x + t(y-x)) $

Which gives:

$g(0) = f(x) \qquad g(1)=f(y)$

The proof starts by using the Fundamental Theorem of Calculus to write:

$f(y) - f(x) = \int_0^1 \langle \nabla f(x+t(y-x)), y-x\rangle dt $

However I don't get why there's a inner product in the integration, to my understanding shouldn't it be:

$f(y) - f(x) = \int_0^1 \nabla f(x+t(y-x)) (y-x) dt $

  • $\begingroup$ $x,y \in \Bbb R^n$. How do you interpret $\nabla f \cdot (y-x)$ if not as an inner product? $\endgroup$ – bames Feb 23 '18 at 10:40

Your notation is unclear. Anyway, by the chain rule, $$ \frac{d}{dt} f(x+t(y-x)) = \nabla f(x+t(y-x)) \cdot (y-x), $$ where the dot denotes the inner product.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.