0
$\begingroup$

I am looking for an activation function that squashes $\mathbb{R}$ to $[0,100]$.

Currently I am using $$f(x) = \frac{100}{1+e^{-x}}$$ but this does not evenly distribute the values across the interval, they are skewed to both ends. By this I mean that all values above $x=5$ map to $\approx 100$ and all values below $x=-5$ map to $\approx 0$, which we can see if we plot the graph. I would like a function which widens the range of inputs which map to values in $(0,100)$.

Can anyone point me towards such a function please.

$\endgroup$
  • $\begingroup$ It's unclear what you mean by "skewed to both ends". Could you please elaborate? $\endgroup$ – Adrian Keister Sep 20 '18 at 16:46
  • $\begingroup$ @AdrianKeister I have updated the question with what I mean $\endgroup$ – lioness99a Sep 21 '18 at 7:48
2
$\begingroup$

Modify your activation function by introducing a scale parameter $b$: $$ f(x)=\frac{100}{1+e^{-x/b}} $$ The larger $b$ is, the more spread out is the range of $x$ values that avoid getting slammed into the extremes of the interval $(0,100)$. Experiment with $b$ to get the behavior you want. Alternatively, if possible, scale the values being fed into the activation function to prevent them from getting too large.

$\endgroup$
  • $\begingroup$ Thank you, this did just what I wanted $\endgroup$ – lioness99a Sep 21 '18 at 12:17
2
$\begingroup$

Any continuous increasing function $f: \mathbb R \to [0, 100]$ will "squash" most of $\mathbb R$. Supposing that $f$ surjects onto $(0, 100)$ -- making "maximal use of the available space" -- you have real numbers $r_1 = f^{-1}(0.005)$ and $r_2 = f^{-1}(99.995)$, and the entire infinite real line except for the part $[r_1, r_2]$ is squashed into one ten-thousandth of the interval $[0, 100]$.

$\endgroup$
0
$\begingroup$

Such a function, not "squashing" $\mathbb R$, can be viewed as $100$x a cumulative distribution function of a uniform distribution on $\mathbb R$. However, such a distribution can't exist as is shown here:

Why isn't there a uniform probability distribution over the positive real numbers?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.