I am following a proof that applies Taylor's theorem on this document (http://www.gautampendse.com/software/lasso/webpage/pendseLassoShooting.pdf)

I am not understanding one of the terms explained on the formula (5.1):

Proposition 5.3 (positive semidefinite Hessian implies Convexity). Suppose x is a p × 1 vector and f(x) is a scalar function of p variables with continuous second order derivatives defined on a convex domain D. If the Hessian ∇2f(x) is positive semidefinite for all x ∈ D then f is convex.

Proof. By Taylor’s theorem for all x, x + h ∈ D we can write:

$$ f(x + h) = f(x) + ∇f(x)^Th + \frac{1}{2}h^T∇^2f(x + \theta h)h $$ for some θ ∈ (0, 1).

I don't understand why the term corresponding to the second derivative is: $$ \frac{1}{2}h^T∇^2f(x + \theta h)h $$

And not:

$$ \frac{1}{2}h^T∇^2f(x )h $$

I understand that by θ ∈ (0, 1), if θ = 0 then the term becomes the latter, but I don't understand what happens when θ > 0.


This formula uses the Cauchy form of the remainder. $x + \theta h $ is a number between $x$ and $x+h$. http://mathworld.wolfram.com/CauchyRemainder.html

It comes from the mean value theorem.

  • $\begingroup$ Many thanks! It was driving me crazy! $\endgroup$ – user1064285 Jan 28 '15 at 10:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.