Are there any examples of solving for the global maximum of a non-differentiable function where you:

  1. Construct a series of differentiable functions that approach the non-differentiable function in the limit
  2. Show the maximum of each differentiable function converges to some value, which is thus your answer.

For all I know, the procedure above is fatally flawed (or there are trivial examples, I would be most interested in non-trivial examples) in some way, if that is the case let me know.

I am specifically interested in examples involving absolute values.

  • $\begingroup$ Could you please clarify the question? Are you looking for examples or counter-examples? $\endgroup$ – Tomer Vromen Aug 5 '10 at 19:45
  • $\begingroup$ Examples, let me know what was confusing and I will edit the question. $\endgroup$ – Jonathan Fischoff Aug 5 '10 at 19:53

A simple example:

Let $F_n(x) = \sqrt{x^2+2^{-n}}$. It is not hard to show that $F_n(x) \to \sqrt{x^2} = |x|$. Every $F_n$ is differentiable and has a local minimum at 0, and indeed so does |x|.

Let me know if this is what you're looking for.

  • $\begingroup$ That is not what I wanted because the right and left derivatives are different for $\sqrt{x^2}$ at zero just like $\left | x \right |$. $\endgroup$ – Jonathan Fischoff Aug 5 '10 at 20:04
  • $\begingroup$ $\sqrt{x^2}$ is $|x|$. Every function in the sequence $F_n$ is differentiable at 0, and the sequence approaches the limit $|x|$. $\endgroup$ – Tomer Vromen Aug 5 '10 at 20:13
  • $\begingroup$ That maybe so, but is every $F_n$ a differentiable function, i.e. has a derivative for each point in its domain? $\endgroup$ – Jonathan Fischoff Aug 5 '10 at 20:20
  • $\begingroup$ I honestly don't know that's why I am asking. $\endgroup$ – Jonathan Fischoff Aug 5 '10 at 20:33
  • $\begingroup$ Okay, my bad. I got out the pen and paper, and it looks you won't get a singularity. $\endgroup$ – Jonathan Fischoff Aug 5 '10 at 21:10

The approach you outlined is commonly used in practice. If your original problem has some nice properties, such as convexity, the approach will work well. For example, the soft maximum is a common way to construct a series of smooth, convex approximations to the maximum function.

  • $\begingroup$ Ha, I actually read part of your blog, but didn't realize the soft maximum approached the function in the limit. Was actually trying to help solve this problem: metaoptimize.com/qa/questions/1715/… which may or not be a good use of the soft maximum $\endgroup$ – Jonathan Fischoff Aug 5 '10 at 21:30

Here's a stab at why the problem is hard for highly non-differentiable but continuous functions. So say $f$ is nowhere differentiable on $[a,b]$. I claim that there are either infinitely or no local extrema of $f$. (By local extrema, I mean extrema that occur in the interior of the interval $[a,b]$.)

Indeed, suppose there were finitely many, say $c_1 < c_2<\dots f(c_2)$. Then we can take the global maximum of $f$ on $[c_1, c_2]$, which occurs at an interior point; it is thus a local extremum of $f$.

If $f$ is a monotone function, then it is a theorem of Lebesgue that it is differentiable a.e. In particular, the above reasoning shows that the existence of finitely many local extrema implies that $f$ is differentiable a.e.

  • $\begingroup$ @Jonathan: I was talking about extrema not at the endpoints. $\endgroup$ – Akhil Mathew Aug 5 '10 at 21:23
  • $\begingroup$ What if you know it has a global maximum, does that change anything? $\endgroup$ – Jonathan Fischoff Aug 5 '10 at 21:37
  • $\begingroup$ are you assuming that $f$ is continuous? $\endgroup$ – Pete L. Clark Aug 6 '10 at 5:04
  • $\begingroup$ Yes, I was (edited again for clarity!). $\endgroup$ – Akhil Mathew Aug 6 '10 at 11:44
  • $\begingroup$ I know this is an old post, but the second paragraph needs work. You probably did not mean $c_2<\dots f(c_2)$. And the part about maximum on $[c_1,c_2]$ being interior is somewhat unclear. (There was a question about it recently: math.stackexchange.com/q/539563) $\endgroup$ – user103402 Oct 29 '13 at 3:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.