0
$\begingroup$
sigmoid > sigmoid(x): 1/(1+np.exp(-x))
derivate > deriv(x): sigmoid(x)*(1-sigmoid(x))

x = 10

sigmoid = 0.999954602131
derivate = 0.196616057953

I think I am correct about calculations. But can someone please explain to me, like to a five year old what derivate value tells us at all? I just know it is a change of something, but change of what? What does 0.196616057953 has to do with 0.999954602131?

What confuses me even more... If i put any number from -x to x, derivate is always about 0.19 up to 0.25 What is the point of this derivate?

I would appreciate your reply. Please do not reply in equations, because I can't not read those symbols :)

$\endgroup$
  • 1
    $\begingroup$ Something may be wrong with your implementation: $0.999954602131 \times (1-0.999954602131) \approx 0.0000453958$ so when $x=10$ the derivative is far from $0.19$. The derivative is between $0.19$ and $0.25$ roughly when $-1.0718 \le x \le 1.0718$ but not otherwise $\endgroup$ – Henry Mar 1 '18 at 17:20
  • 1
    $\begingroup$ It tells you the rate of change. Geometrically, what is the slope of the tangent line (numerically, it's a secant line unless you have the exact form of the derivative) to the sigmoid at a point? $\endgroup$ – Adrian Keister Mar 1 '18 at 18:19
1
$\begingroup$

The derivative is to the function what speed is to distance.

To make it simpler, for $x=10$ you got $sigmoid = 0.999954602131$.

Repeat the calculation for $x=9.9$ to get $sigmoid = something$.

Now compute $$\frac{ 0.999954602131-something}{10 -9.9}=10(0.999954602131-something)$$

This should look like the number you got for $derivate$

Edit

After Henry's comment, what it seems is that you computed $\text{sigmoid}(10)=0.9999546021$ but $\text{deriv}(1)=0.1966119332$ while $\text{deriv}(10)=0.0000453958$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.