# Initialization of Limited-memory BFGS (using libLBFGS)

I am using the package libLBFGS in order to minimize an objective function, for which the first derivative (with respect to the optimization variable) is known and computable. I use the default parameters, as shown in the sample code of the above webpage. The code runs and the optimal solution is obtained. The problem lies in the initialization phase. Although for some initial values the optimal solutions are suitable, for different ones (even just a bit different) the optimal solutions differ, sometimes a lot. I would like to ask the following:

a) If the above problem concerns the convexity of the optimization function, could I prove that the LBFGS algorithm will or will not converge?

b) Has anyone else used that library (libLBFGS) for optimizing unconstrained, non-linear before? I am not sure, but I think I am missing something in tuning the algorithm via the API of the library. The documentation of the API is not clear enough, I think...

c) Would you say that implementing the LBFGS from scrath is a feasible/rational choice?

Thanks a lot!

• Have you checked that your derivatives are correct? Is your cost convex? – copper.hat Dec 20 '13 at 16:37
• My derivatives seem to be correct. By 'cost' you mean the whole objective function, or a part of it. In both cases, I am not sure whether the function is convex or not. But if we say that it's not convex, does the situation above seems reasonable (different initialization --> different solution)? – nullgeppetto Dec 20 '13 at 16:46
• In general, yes (take $x \mapsto \sin x$ for example). But it is impossible to say without knowing the details. Your application will have some notion of scale, in general if initial points are 'close' you would expect the results to be 'close', purely because BFGS is usually a descent method (I do not know the specifics of the LBFGS method). – copper.hat Dec 20 '13 at 16:52