# Non-convex numerical optimization

I am looking for a way how to prove/show that one problem is easier to optimize(more robust to starting guesses) than another.

Suppose I have two non-convex functions from $$L_1,L_2:\mathbb{R_+}^8\rightarrow\mathbb{R}$$, and I am trying to numerically find their minimum on an unconstrained support $$\mathbb{R}_+$$.

Specifically I am dealing with two types of log-likelihood functions for the same data. However, where $$L_2$$ seems agnostic to starting points, $$L_1$$ converges to different local minima.

I have investigated contour subplots of $$L_1$$, i.e. keeping all parameters fixed except varying two of them. On the plot below black X are starting points, blue diamonds are points marked as local optima by the solver and red points are those points where optimizer converged to absurdly large values.

I notice that $$L_2$$ does not exhibit this behaviour (even though structurally is very similar to $$L_1$$) and therefore is more robust to the choice of starting guesses. I would like to get an idea why this happens. I have noticed that when comparing the two functions in some subspace of $$\mathbb{R}^8$$, $$L_2$$ is more "curved" than the other, as shown in a plot bellow.

Can this be the reason and if so is there a mathematical (even numerical) way to show that one is more curved at a particular point than the other (perhaps using Hessian)?

Thanks!

• Your plot for $L_1$ seems to be levelling off, even bending over. Does this function have another minima by any chance? or do you consider $\delta < \varepsilon$ change as converged? – Chinny84 Feb 6 at 15:22
• Exactly, the function is bending over implicating its non-convexity. $L_1$ is confirmed to have multiple local minima in the full $\mathbb{R}^8$ space -- different starting guesses result in different points of convergence. By convergence I mean that the step size is either sufficiently small or step in any direction does not improve the obj. function. Thanks. – Michael Mark Feb 6 at 16:58
• Ok, I am assuming that $L_1$ bends over in the vicinity of the initial conditions. But I am not completely sure, so I will default to another poster. – Chinny84 Feb 6 at 17:06