Derivative and second derivative of scaled logistic I am implementing some custom loss functions for an AI project and my calculus is very rusty...
I need the first and second derivative of variants of the scaled logistic/sigomoid function $S(x)=\frac{1}{1+e^{-kx}}$
According to this link and others, the first derivative of $S(x)$ when $k=1$ is $S'(x) = S(x)(1-S(x))$, and the second derivative: $S''(x) = S(x) (1-S(X)) (1-2S(X))$, but how does changing $k$ affect this result?
 A: Sorry for this very late answer, but let's differentiate!

We use the chain rule:
$$F'(x)=f'(g(x))g'(x),$$
where $F'(x) = f(g(x))$ and $f'(x)$ is the first derivative of $f(x)$.

So let's start with the first derivative:
$$S(x)=\frac{1}{1+e^{-kx}}=(1+e^{-kx})^{-1}$$
Let's denote $f(x)=x^{-1}$ and $g(x)=1+e^{-kx}$
Let's calculate:
$$g'(x) = \frac{d}{dx}(1+e^{-kx})=\frac{d}{dx}1+\frac{d}{dx}e^{-kx}=0+h(x)=h'(x)$$
where $h(x)=e^{-kx}$.
Let's calculate:
$$h'(x)=\frac{d}{dx}e^{-kx}$$
Let's denote $i(x)=e^x$ and $j(x)=-kx$
Let's move on:
$$j'(x)=\frac{d}{dx}(-kx)=-k\frac{d}{dx}x=-k$$
$$i'(x)=\frac{d}{dx}e^x=e^x$$
By the chain rule:
$$h'(x)=i'(j(x))j'(x)=-ke^{-kx}$$
Then,
$$g'(x)=h'(x)=-ke^{-kx}$$
$$f'(x)=-x^{-2}$$
By the chain rule,
$$S'(x)=f'(g(x))g'(x)=-(1+e^{-kx})^{-2}\cdot (-ke^{-kx})=\frac{ke^{-kx}}{(1+e^{-kx})^2}$$

Now the second derivative:
$$S''(x)=\frac{d}{dx}S'(x)=\frac{d}{dx}\frac{ke^{-kx}}{(1+e^{-kx})^2}=\frac{d}{dx}\frac{a(x)}{b(x)}$$
With the quotient rule:
$$\frac{d}{dx}\frac{a(x)}{b(x)}=\frac{a'(x)b(x)-b'(x)a(x)}{b(x)^2}$$
$$a'(x)=\frac{d}{dx}(ke^{-kx})=k\cdot h'(x)=-k^2e^{-kx}$$
$$b'(x)=\frac{d}{dx}(1+e^{-kx})^2=\frac{d}{dx}c(d(x))$$
where $c(x)=x^2$ and $d(x)=1+e^{-kx}$
$$c'(x)=\frac{d}{dx}x^2=2x$$
$$d'(x)=g'(x)=-ke^{-kx}$$
By the chain rule,
$$b'(x)=c'(d(x))d'(x)=2(1+e^{-kx})\cdot(-ke^{-kx})=-2ke^{-kx}(1+e^{-kx})$$
With the quotient rule,
$$S''(x)=\frac{d}{dx}\frac{a(x)}{b(x)}=\frac{a'(x)b(x)-b'(x)a(x)}{b(x)^2}=\frac{-k^2e^{-kx}\cdot (1+e^{-kx})^2 - (1+e^{-kx})\cdot ke^{-kx}}{(-2ke^{-kx}(1+e^{-kx}))^2}=4e^{kx}+\frac{e^{kx}}{4k(1+e^{-kx})}=-\frac{k^2(e^{kx}-1)e^{kx}}{(e^{kx}+1)^3}$$
So this is all.
P.S. Maybe it is very long, but I have included every step. Feel free to correct me if I am wrong.
Happy differentiating!
A: I'll summarize in a succinct form.
For general $k$, $S'(x) = k S(x) (1-S(x))$, and $S''(x) = k^2 S(x) (1-S(x)) (1-2S(x))$.
For an even more general logistic function $S_C(x)=\frac{C}{1+e^{-kx}}$ with magnitude $C$, the derivatives are $S_C'(x) = (\frac{k}{C}) S_C(x) (C-S_C(x))$, and $S_C''(x) = (\frac{k}{C})^2 S_C(x) (C-S_C(x)) (C-2S_C(x))$.
Shifting of $x \to x - \mu$ does not affect these results.
