Differences behind different methods of fminunc in MATLAB?

Question

Assume I have some .m file with a function (and it's gradient) to be used by fminunc() in MATLAB for some unconstrained optimization problem.

To solve the problem in the most simple way, I do this:

clear all
[x,fval] = fminunc(@fun, [1;1])

This will minimize fval and return the optimized values of x. For a more accurate optimization, I do this:

clear all
op = optimset('GradObj', 'on', 'LargeScale', 'off');
[x,fval] = fminunc(@fun, [1;1], op)

Both fval and x values still are the solution to the problem only that now they are more accurate, because of the supplied gradient. Correct?

Both of the above methods use the line-search algorithm but I can also use the trust-region algorithm, like this:

clear all
op = optimset('GradObj', 'on');
[x,fval] = fminunc(@fun, [1;1], op)

Both fval and x values are different from the previous ones. What does this mean? Is this algorithm better or worse? Or maybe it's different in a way that it's not better nor worse. What does it mean than?

Answer 1

The main benefit of providing a gradient is that convergence should be quicker. In general, even without gradient, you should be able to converge to a solution to within a desired accuracy, although at the expense of more iterations. A more direct control over accuracy is given by options 'TolX' and 'TolFun'.

If the objective functions have local minima, the solution you find will naturally depend on initial conditions. Differences in algorithms cause them to follow different paths in their way to the solution, so when applied to functions having many local minima, they may end up in different solutions.

In minimization problems, it's actually easy to choose the best among different solutions: just choose the one with the lower value! :-) Their meaning depends, of course, on the actual problem being solved...