5
$\begingroup$

I've just read the following

The basic unit ("neuron" i) performs the following computation to update its state $y_i$: it computes a weighted sum $v_i$ of its inputs $x:j$ which is passed through a sigmoid squashing function $g ( \cdot )$.

Source: Design of a neural network character recognizer for a touch terminal

I know what a sigmoid function is, but what is a sigmoid squashing function? I have also seen this in the PyBrain documentation.

$\endgroup$
4
  • $\begingroup$ It's basically a sigmoid function used to compress the outputs. $\endgroup$
    – Silynn
    Jun 18, 2014 at 21:32
  • $\begingroup$ So is "squashing function" a synonym for "activation function"? $\endgroup$
    – Martin Thoma
    Jun 18, 2014 at 21:35
  • $\begingroup$ So is it "a squashing function that we chose to be a sigmoid function" or is it "a special type of sigmoid function that is called 'squashing'"? $\endgroup$
    – Martin Thoma
    Jun 18, 2014 at 21:36
  • $\begingroup$ They just mean that they squash the signal (saturate it), using a sigmoid function (as you could use other functions for squashing). $\endgroup$
    – user65203
    Jun 18, 2014 at 22:11

2 Answers 2

Reset to default
6
$\begingroup$

The sigmoid squashing function is the same as the sigmoid function.

The term sigmoid squashing function is favored in the neural net community. The logistic function is the classical squashing function. Other sigmoid functions include: arctangent, the hyperbolic tangent, the Gudermannian function, and the error function.

It is called the squashing function because it is tightly bounded to the pair of horizontal asymptotes. Thus, useful in compressing, or squashing, outputs.

$\endgroup$
1
  • $\begingroup$ Do you mean something like this? Given an input, its effect is (let's say) linear when the activation to which it will contribute is (e.g.) near 0 (for activations in [-1,1]), but when the activation is near an extremum (1, let's say), then the effect of the input is reduced--i.e. it is "squashed". $\endgroup$
    – Mars
    Jan 26, 2015 at 19:14
1
$\begingroup$

Fix a reference measure $\mu$.
A sigmoid function $\sigma_0$ is a essentially-bounded map from $\mathbb{R}$ to itself. I.e.: $\sigma_0 \in L^{\infty}_{\mu}(\mathbb{R})$.

While a squashing function $\sigma$ is a sigmoid function satisfying $$ \operatorname{ess-sup}|\sigma| \leq 1 , $$ thus, $\sigma \in Ball_{L^{\infty}_{\mu}(\mathbb{R})}$.

Practical Lingo:

In practical terms, if $\mu$ is the Lebesgue measure, then $\sigma_0:\mathbb{R}\to [m,M]$ for some $m\leq M$, real numbers, while $\sigma:\mathbb{R} \to [-1,1]$.

I think this paper has a rigerous definitio (under the additional assumption of continuity). While this paper hints to the non-continuous version.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .