# Sigmoid-Normal Inequality

I conjecture that for any $$\beta>0$$, \begin{align} \sigma(\beta) > \Phi(x_{\beta}) \quad\text{where}\quad x_{\beta} = \beta \phi(x_{\beta}). \end{align}

• $$\sigma$$ denotes the sigmoid function defined by $$\sigma(x) = \frac{1}{1+e^{-x}}$$.
• $$\Phi$$ and $$\phi$$ denote the standard normal cdf and pdf, respectively.

The conjecture seems to be true according to my numerical analysis, but I am unsure about a formal proof.

I would appreciate it if you would share your thoughts or suggestions. Thank you.

• $x_{\beta} = \beta \phi(x_{\beta})$ looks like a circular definition Commented Mar 28 at 2:14
• Thank you for your comment. The definition is not circular, however. The equation $x = \beta \phi(x)$ has a unique solution, $x_{\beta} > 0$ for all $\beta > 0$. Commented Mar 28 at 3:11
• It is true for $\beta >0$. From your $x_{\beta} = \beta \phi(x_{\beta})$, consider $b(x)=\frac{x}{\phi(x)}$ which is a bijection $\mathbb R \to \mathbb R$ which preserves signs so has an inverse. You can say $\beta=b(x_\beta)$, i.e. $x_\beta=b^{-1}(\beta)$. Your assertion becomes that for any $x_\beta>0$ you have $\dfrac{1}{1+e^{-x_\beta/\phi(x_\beta)}} >\Phi(x_\beta)$, which is true - to prove it you might want to separate into small and large $x_\beta$ and find simple functions between the LHS and RHS. Commented Mar 28 at 12:53

## 1 Answer

We rephrase the problem as follows.

Problem. Prove that, for all $$\beta > 0$$, $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \int_{-\infty}^{u} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t$$ where $$u$$ is a real number satisfying $$u = \beta \cdot \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-u^2/2}.$$ (Note: Clearly, such $$u$$ exists (unique) and is positive.)

$$\phantom{2}$$

Proof.

We split into two cases.

Case 1. $$\beta \ge 3$$

Let $$f(u) := u - \beta \cdot \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-u^2/2}$$. We have $$f'(u) > 0$$. We have $$f(\sqrt{2\ln \beta}) = \sqrt{2\ln \beta} - \frac{1}{\sqrt{2\pi}} \ge \sqrt{2\ln 3} - \frac{1}{\sqrt{2\pi}} > 0 = f(u)$$ which results in $$\sqrt{2\ln \beta} \ge u$$. It suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \int_{-\infty}^{\sqrt{2\ln \beta}} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{1}$$

We have, for all $$u>0$$ $$1 - \frac{u^2 - 1}{u^3\sqrt{2\pi}}\,\mathrm{e}^{-u^2/2}\ge \int_{-\infty}^{u} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{2}$$ (Proof: Let $$g(u) := \mathrm{LHS} - \mathrm{RHS}$$. We have $$g'(u) = - \frac{3}{u^4\sqrt{2\pi}}\mathrm{e}^{-u^2/2}< 0$$ for all $$u > 0$$. Also, $$g(\infty) = 0$$. Thus, $$g(u)\ge 0$$ for all $$u > 0$$.)

From (1) and (2), it suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > 1 - \frac{2\ln \beta - 1}{\sqrt{2\ln \beta}^3\sqrt{2\pi}}\cdot \frac{1}{\beta}. \tag{3}$$

We have \begin{align*} &\mathrm{e}^{-\beta} = \frac{1}{\mathrm{e}^3\mathrm{e}^{\beta-3}} \le \frac{1}{\mathrm{e}^3[1 + (\beta - 3) + \frac12(\beta - 3)^2]}, \tag{4}\\[6pt] &\frac{2\ln \beta - 1}{\sqrt{2\ln \beta}^3\sqrt{2\pi}} = \frac{1 - 1/(2\ln\beta)}{\sqrt{2\ln \beta}\sqrt{2\pi}} \ge \frac{1 - 1/(2\ln 3)}{\sqrt{2\ln \beta}\sqrt{2\pi}},\tag{5}\\[6pt] &\ln \beta = \ln 3 + \ln (1 + (\beta - 3)/3) \le \ln 3 + (\beta - 3)/3. \tag{6} \end{align*}

From (3)-(6), it suffices to prove that $$\frac{1}{1 + \frac{1}{\mathrm{e}^3[1 + (\beta - 3) + \frac12(\beta - 3)^2]}} > 1 - \frac{1 - 1/(2\ln 3)}{\sqrt{2\ln 3 + 2(\beta - 3)/3}\cdot \sqrt{2\pi}}\cdot \frac{1}{\beta},$$ or $$\frac{1 - 1/(2\ln 3)}{\sqrt{2\ln 3 + 2(\beta - 3)/3}\cdot \sqrt{2\pi}}\cdot \frac{1}{\beta} > \frac{2}{\mathrm{e}^3(\beta^2 - 4\beta + 5) + 2},$$ or $$\frac{(1 - 1/(2\ln 3))^2}{[2\ln 3 + 2(\beta - 3)/3]\cdot 2\pi}\cdot \frac{1}{\beta^2} > \frac{4}{[\mathrm{e}^3(\beta^2 - 4\beta + 5) + 2]^2},$$ or (using $$\mathrm{e}^3 > 20$$ and $$2\ln 3 > 2 + 1/6$$ and $$2\ln 3 < 2 + 1/5$$ and $$2\pi < 6 + 1/3$$) $$\frac{(1 - 1/(2 + 1/6))^2}{[(2 + 1/5) + 2(\beta - 3)/3]\cdot (6 + 1/3)}\cdot \frac{1}{\beta^2} > \frac{4}{[20(\beta^2 - 4\beta + 5) + 2]^2},$$ or $${\frac {1}{3211}}\,{\frac {220500\,{\beta}^{4}-1796110\,{\beta}^{3}+5767467\,{ \beta}^{2}-8996400\,\beta+5735205}{ \left( 3+10\,\beta \right) {\beta}^{2} \left( 10\, {\beta}^{2}-40\,\beta+51 \right) ^{2}}} > 0$$ which is true.

$$\phantom{2}$$

Case 2. $$0 < \beta < 3$$

Let $$f(u) := u - \beta \cdot \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-u^2/2}$$. We have $$f'(u) > 0$$. Using $$\sqrt{2\pi} > 5/2$$ and $$\mathrm{e}^{-2\beta^2/25} = \frac{1}{\mathrm{e}^{2\beta^2/25}} \le \frac{1}{1+ 2\beta^2/25}$$, we have $$f(2\beta/5) = \frac{2\beta}{5} - \frac{\beta \mathrm{e}^{-2\beta^2/25}}{\sqrt{2\pi}} \ge \frac{2\beta}{5} - \frac{ \beta \cdot \frac{2}{5}}{1+ 2\beta^2/25} = \frac{4\beta^3}{10\beta^2 + 125} > 0 = f(u)$$ which results in $$2\beta/5 \ge u$$. It suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{7}$$

(i) If $$11/4 \le \beta < 3$$, we have $$\frac{1}{1 + \mathrm{e}^{-\beta}} - \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t \ge \frac{1}{1 + \mathrm{e}^{-11/4}} - \int_{-\infty}^{2\cdot 3/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t > 0.$$

(ii) If $$0 < \beta < 11/4$$, it suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \frac{2\beta}{5\sqrt{2\pi}} + \frac12 > \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{8}$$

Let $$F(\beta) := \frac{1}{1 + \mathrm{e}^{-\beta}} - \frac{2\beta}{5\sqrt{2\pi}} - \frac12$$. We have $$F''(\beta) := - \frac{\mathrm{e}^\beta(\mathrm{e}^\beta - 1)}{(1 + \mathrm{e}^\beta)^3} > 0$$. Also, $$F(0) = 0$$ and $$F(11/4) > 0$$. Thus, $$F(\beta) > 0$$ on $$(0, 11/4)$$.

Let $$G(\beta) := \frac{2\beta}{5\sqrt{2\pi}} + \frac12 - \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t$$. We have $$G'(\beta) = \frac{2(1 - \mathrm{e}^{-2\beta^2/25})}{5\sqrt{2\pi}} > 0$$. Also, $$G(0) = 0$$. Thus, $$G(\beta) > 0$$ on $$(0, 11/4)$$.

Thus, (8) is true.

We are done.

• Thank you very much. It is perfect. May I ask (from my curiosity) how you came up with the series of ingenious inequalities? Commented Mar 29 at 23:18
• @Pierre For example, LHS of (2) is a result from the asymptotic expansion of the RHS of (2). (8) is to find a linear segment separating the two curves. We can do it with the help of Maple - a computer algebra system. Commented Mar 30 at 0:24