5
$\begingroup$

I saw the following result: $$ \dfrac{\mathrm{d}}{\mathrm{d}x} \left( \log\left( \dfrac{1}{1+\mathrm{e}^{-x}} \right) \right) = \dfrac{1}{\mathrm{e}^x+1} $$ What are the intermediary steps for obtaining this result?

$\endgroup$
  • $\begingroup$ Where are you stuck? $\endgroup$ – StubbornAtom Jun 13 '17 at 11:44
  • $\begingroup$ So I first differentiate the logarithm, according to the rule $log(x)' = 1/x$ and then I differentiate the rest. From the first step I get a $1 + e^{-x}$ which I can not then simplify. $\endgroup$ – octavian Jun 13 '17 at 11:55
5
$\begingroup$

Hint:

First, notice that $$ \begin{align} \dfrac{1}{1+e^{-x}} = \dfrac{\mathrm{e}^{x} \cdot 1}{\mathrm{e}^{x} \cdot 1 + \mathrm{e}^{x} \cdot e^{-x}} = \dfrac{\mathrm{e}^{x}}{\mathrm{e}^{x} + 1} \;. \end{align} $$

Second, notice that $$ \begin{align} \ln\left( \dfrac{\mathrm{e}^{x}}{\mathrm{e}^{x} + 1} \right) = \ln\left( \mathrm{e}^{x}\right) - \ln\left( \mathrm{e}^{x} + 1 \right) = x - \ln\left( \mathrm{e}^{x} + 1 \right) \;. \end{align} $$

So, we have $$ \begin{align} \dfrac{\mathrm{d}}{\mathrm{d}x} \log\left( \dfrac{1}{1+\mathrm{e}^{-x}} \right) &= \dfrac{\mathrm{d}}{\mathrm{d}x} \left( x - \ln\left( \mathrm{e}^{x} + 1 \right) \right) = \dfrac{\mathrm{d}x}{\mathrm{d}x} - \dfrac{\mathrm{d}\ln\left( \mathrm{e}^{x} + 1 \right)}{\mathrm{d}x} \;. \end{align} $$

Can you go on from here using the chain rule?

| cite | improve this answer | |
$\endgroup$
5
$\begingroup$

You just have to use the Chain Rule.

$\alpha = 1+e^{-x}$

$\beta = \alpha^{-1}$

$\frac{d\,log(\beta)}{d\,x} = \frac{d\,log(\beta)}{d\,\beta}\,\frac{d\,\beta}{d\,x} = \frac{d\,log(\beta)}{d\,\beta}\,\frac{d\,\alpha^{-1}}{d\,\alpha}\,\frac{d\,\alpha}{d\,x} = \left(\frac{1}{\beta}\right)\,\left(-\frac{1}{\alpha^2}\right)\,\left(-e^{-x}\right) = \frac{e^{-x}}{1+e^{-x}} = \boxed{\frac{1}{e^x + 1}}$

You don't have to worry with signs, because everything in there is always strictly positive.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

Log base could refer different bases for different fields. Notice that log(x) refers to base-2 log for computer science, base-e log for mathematical analysis and base-10 log for logarithm tables.

In most general form, derivative of y = logb(1/(1 + ex)) is in following form:

dy/dx = 1 / (ln(b) . (1 + ex))

Of course, if main function were refered to natural logarithm, then b would equal to e, and derivative would be:

dy/dx = 1 / (ln(e) . (1 + ex))

ln(e) would be 1 based on the logarithm of the base rule.

dy/dx = 1 / ((1 + ex))

Mostly, natural logarithm of sigmoid function is mentioned in neural networks. Activation function is calculated in feedforward step whereas its derivative is calculated in backprogation. And derivative of natural log of sigmoid is easier to calculate than other bases.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.