4
$\begingroup$

I've been working on a project to maximize a functions output using a genetic algorithm. However, from the limited calculus I know I thought there were methods to find the maximum of a mathematical function using calculus? I'd assume the reason genetic algorithms are sometimes used to maximize functions is because there are functions where the mathematical methods don't work. If I'm correct, what are those conditions? I'd guess that maybe it's because the function is not continuous or differentiable?

$\endgroup$
  • $\begingroup$ Not related too much to the question, but I think you might find it interesting. en.wikipedia.org/wiki/No_free_lunch_in_search_and_optimization $\endgroup$ – AnotherUser Aug 4 '14 at 19:20
  • 2
    $\begingroup$ Many times you want to minimize a nondifferentiable function. Even if your objective function is differentiable, there may be many local minima. When you set the derivative equal to $0$, there may not be an analytical formula that tells you all the solutions. So it's not as if you can just set the derivative equal to $0$ and solve for $x$.You need to use some strategy like gradient descent, which typically leads you to a local minimum. $\endgroup$ – littleO Aug 4 '14 at 20:39
  • $\begingroup$ @littleO So you can't use formula to get a global maximum from a nondifferentiable function? You need to use a search strategy such as a genetic algorithm? I recall formulas that could maximize some functions with local optimums. Also, gradient descent would require the function to be differentiable. $\endgroup$ – user11406 Aug 4 '14 at 21:02
  • 1
    $\begingroup$ If you are trying to minimize a function which is nondifferentiable but which is convex, then there's a good chance that numerical methods from convex optimization would allow you to compute a global minimizer efficiently. But rarely is there an actual formula for the solution to an optimization problem that arises in practice. $\endgroup$ – littleO Aug 4 '14 at 21:20
  • 1
    $\begingroup$ I haven't seen a lot of interest in genetic algorithms to solve single-variable functions. Mostly it's functions of many variables. Local optimums of such functions can be a lot harder to find. $\endgroup$ – David K Aug 4 '14 at 21:21
1
$\begingroup$

Calculus methods are useful for functions which are differentiable. For functions which are not differentiable calculus won't help much. For instance the data set could be discrete. Some examples of this include:

  1. A set of binary variables like a set of yes/no decisions. Which set of filters and parameters shall we use for this image to get best image quality?
  2. Which order to visit cities will minimize cost? Or which vehicles to transport which goods to minimize time and/or required resources?
  3. Which set of coins will minimize time to pay a beer in cash?
$\endgroup$
0
$\begingroup$

First off, the calculus techniques for determining the optimum value of a function are used, however analytic techniques sometimes fall short. Classic examples are if the function isn't continuous, but it's also possible you are optimizing against an extremely large function (logistic regression, where you are minimizing the function which has 1 coefficient per input, and say a few million inputs. Genetic algorithms are often applied in a similar place, where analytic techniques can't resolve an answer and iterative methods (such as newton's method, although that's not actually used), have to be applied.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.