3
$\begingroup$

I need help. In my class we're not using Hessian matrices with our convex optimization. I don't really understand how to find out if it's concave or convex. All the videos I look at and articles I read pretty much only use Hessian Matrices and what not.

I don't really understand this Convex Theorem either. f(λx+(1−λ)y)≥λf(x)+(1−λ)f(y). f(λx+(1−λ)y)≥λf(x)+(1−λ)f(y) f(λx+(1−λ)y)≤λf(x)+(1−λ)f(y) It seems like it's saying the function should be larger than the extrema for concavity and less than for convexity?

I get what concave and convex mean, I looked at a fantastic video that explained where they derived that Convex theorem equation, and I get that you should use the second derivative. But I don't really understand how you find out or use the second derivative.

I'm terribly sorry. Calculus was several years ago. Do you think someone could please help me out? I know this has to be a very annoying and simple question for you guys...

$\endgroup$
  • $\begingroup$ Humility is important, but I don't think this is annoying or simple at all. No need to fret. Unfortunately I know very little about the subject myself, or I'd jump in. $\endgroup$ – The Count Mar 15 '17 at 17:36
  • $\begingroup$ Those inequalities say that if you have two points $(x,f(x)),(y,f(y))$ on the curve of a convex function, the graph of the convex function between those points lies below the line between them. It's reversed for a concave function. Try sketching this out for $f(x)=x^2$ (a convex function) and $f(x)=\sqrt{x}$ (a concave function). $\endgroup$ – Ian Mar 15 '17 at 18:08
  • 1
    $\begingroup$ Deriving the Hessian is one way to prove convexity, but it is not the only way, and of course it's useless for nondifferentiable functions. I recommend the book Convex Optimization by Boyd & Vandenberghe (free downloadable available) as a good text. That said, if you struggle with vector calculus, I'm afraid you're going to have quite a difficult time navigating any decent text on convex optimization. $\endgroup$ – Michael Grant Mar 15 '17 at 18:12
  • $\begingroup$ As for the role of Hessian matrices, in the 1D case the idea is that if the first derivative is zero and the second derivative is positive (resp. negative), then whichever way you go from the point, the value goes up (resp. down). In higher dimensions we have more directions than just left and right, but the principle is still the same: we want to guarantee that whatever direction we go from our minimum (resp. maximum) point, the function values go up (resp. down). Seeking out a point where the gradient vanishes and the Hessian is positive (resp. negative) definite is how we do this. $\endgroup$ – Ian Mar 15 '17 at 18:14
  • $\begingroup$ @TheCount Thank you for saying that. I always worry I bother people. $\endgroup$ – MorbidTag Mar 15 '17 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.