# Do BFGS and L-BFGS methods converge when the matrix $H=B^{-1}$ is ill-conditioned?

I am using BFGS and L-BFGS to solve an unconstrained optimization problem. The objective function is the Mean Euclidean Error. The output is given by an Artificial Neural Network. The line search produces step sizes $$\alpha$$ that satisfy the Wolfe conditions.

The methods struggle to converge to a minimizer $$w^*$$ when the approximation to the inverse of the Hessian, $$H = B^{-1}$$, is ill-conditioned, that is it's condition number $$k(H) = ||H||_2 ||H^{-1}||_2 = \sigma_1/\sigma_n$$ is high.

Here are the plots of gradient norm, relative gap error, step size, condition number:

• Stop criterion: $$||\nabla E|| < 10^{-5}$$, as you can see the curves zig-zag a lot.

• this looks fine

• condition number increases