Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Assume I have some .m file with a function (and it's gradient) to be used by fminunc() in MATLAB for some unconstrained optimization problem.

To solve the problem in the most simple way, I do this:

clear all
[x,fval] = fminunc(@fun, [1;1])

This will minimize fval and return the optimized values of x. For a more accurate optimization, I do this:

clear all
op = optimset('GradObj', 'on', 'LargeScale', 'off');
[x,fval] = fminunc(@fun, [1;1], op)

Both fval and x values still are the solution to the problem only that now they are more accurate, because of the supplied gradient. Correct?

Both of the above methods use the line-search algorithm but I can also use the trust-region algorithm, like this:

clear all
op = optimset('GradObj', 'on');
[x,fval] = fminunc(@fun, [1;1], op)

Both fval and x values are different from the previous ones. What does this mean? Is this algorithm better or worse? Or maybe it's different in a way that it's not better nor worse. What does it mean than?

share|improve this question
    
Without saying what your fun() looks like, and the results you're getting, it's hard to say anything except that it is entirely possible for the two methods to return different results. These algorithms optimize locally, so it is entirely possible that the two methods converged to different local optima (if your function has multiple optima). –  J. M. Dec 15 '11 at 0:52
    
I can post the function but why does it matter? I simply want to know the meaning/difference between both algorithms... –  Ricardo Amaral Dec 15 '11 at 1:21
    
Neither is better nor worse. The fact that both algorithms returned different results for the same set of inputs is usually a signal that there's something screwy with your function. As for meaning: they're both modifications of Newton-Raphson for optimization; the difference lies in the safeguards being used to tame the usual divergence seen in Newton-Raphson methods when the starting points are less than stellar. You'll want to look at Dennis/Schnabel if you wish for more details. –  J. M. Dec 15 '11 at 1:32
    
Could you write a proper answer with your comments above so I can accept it? –  Ricardo Amaral Dec 15 '11 at 15:22
    
You'll need to wait a bit; I'm busy with a few other things. –  J. M. Dec 15 '11 at 15:27

1 Answer 1

The main benefit of providing a gradient is that convergence should be quicker. In general, even without gradient, you should be able to converge to a solution to within a desired accuracy, although at the expense of more iterations. A more direct control over accuracy is given by options 'TolX' and 'TolFun'.

If the objective functions have local minima, the solution you find will naturally depend on initial conditions. Differences in algorithms cause them to follow different paths in their way to the solution, so when applied to functions having many local minima, they may end up in different solutions.

In minimization problems, it's actually easy to choose the best among different solutions: just choose the one with the lower value! :-) Their meaning depends, of course, on the actual problem being solved...

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.