# How to get the loss function derivative

I am following a lecture on logistic regression using gradient descent and I have an issuer understanding a short-path for a derivative :

Let be :

• $z=w_1x_1+w_2x_2+b$
• $a=\sigma(z)$
• and the loss function $L(a,y)=-y(\log(a)+(1-y)\log(1-a))$, which I know have a name but I can't remember it it.

In order to have $\frac{\delta\mathcal L(a,y)}{\delta z}$ I am able to compute $\frac{\delta\mathcal L(a,y)}{\delta a}$. But where do we get $\frac{\delta a}{\delta z}=a(1-a)$ ?

I can only have $\frac{\delta a}{\delta z}=\frac{\delta\sigma(w_1x_1+w_2x_2+b)}{\delta z}$

If $$\sigma(x) = \frac{1}{1+\exp{(-x)}}$$ then $$\sigma^\prime(x) = \frac{-1}{[1+\exp(-x)]^2}\cdot(-\exp(-x)) = \frac{\exp(-x)}{[1+\exp(-x)]^2}$$ (by the quotient rule)
And on the other hand, $$\sigma(x)\cdot [1-\sigma(x)]$$ is
\begin{align*} \frac{1}{1+\exp{(-x)}} \cdot \left[1-\frac{1}{1+\exp{(-x)}}\right] &= \frac{1}{1+\exp{(-x)}} \cdot \left[\frac{1+\exp{(-x)}}{1+\exp{(-x)}} - \frac{1}{1+\exp{(-x)}}\right]\\&=\frac{\exp(-x)}{\left[1+\exp(-x)\right]^2}\end{align*}
which is the same thing. So, $$\sigma^\prime(x) = \sigma(x) \cdot [1-\sigma(x)]$$.
$$\sigma(z)=\frac{1}{1+\exp(-z)}$$