7
$\begingroup$

There is a little triviality that has been referred to as the "soft maximum" over on John Cook's Blog that I find to be fun, at the very least.

The idea is this: given a list of values, say $x_1,x_2,\ldots,x_n$ , the function

$g(x_1,x_2,\ldots,x_n) = \log(\exp(x_1) + \exp(x_2) + \cdots + \exp(x_n))$

returns a value very near the maximum in the list.

This happens because that exponentiation exaggerates the differences between the $x_i$ values. For the largest $x_i$, $\exp(x_i)$ will be $really$ large. This largest exponential will significantly outweigh all of the others combined. Taking the logarithm, i.e. undoing the exponentiation, we essentially recover the largest of the $x_i$'s. (Of course, if two of the values were very near one another, we aren't guaranteed to get the true maximum, but it won't be far off!)

About this, John Cook says: "The soft maximum approximates the hard maximum but it also rounds off the corners." This couldn't really be said any better.

I recall trying to cleverly construct sequences for proofs in advanced calculus where not-everywhere-differentiable operations would have been great to use if they didn't have that pesky non-differentiable trait. I can't recall a specific incidence where I was tempted to use $max(x_i)$, but this seems at least plausible that it would have come up.

Has anyone used this before or have a scenario off hand where it would be useful?

$\endgroup$
2
  • $\begingroup$ This is similar to the infinity norm, which is essentially the nth root of the sum of the nth powers of the x_i's letting n tend to infinity in the limit, but more usually defined as the max of the absolute values of the x_i's. However, because of this, its not a true max if you allow negative (x_i's), whereas the above is not affected. $\endgroup$
    – James
    Jul 23, 2010 at 14:34
  • $\begingroup$ You know, for some reason I did not expect John D. Cook to show up here! @John: Welcome! $\endgroup$ Jul 28, 2010 at 19:46

1 Answer 1

3
$\begingroup$

This is close to being the flipside of the geometric mean, which is the nth root of the product of the numbers, and can be expressed as the exponential of the sum of the logarithms.

Another pair of dual mean measures is the regular mean and the harmonic mean (n divided by the sum of the reciprocals).

I say the soft maximum is close to being the flipside of the geometric mean, but it lacks the good property that all of the others have of taking a list of the same value to that value (for definedness, let all values be positive). Let's call the hyperbolic mean the "soft maximum" of the nth roots of the terms in the list: then this has that good property.

The hyperbolic mean the emphasises large values in a roughly symmetric manner to the way that the geometric mean emphasises small values (which is always smaller than the regular mean), and is, of course, much smaller for long list of large values.

So I say, consider it an amplified version of a useful addition to the family of of mean operators.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.