2
$\begingroup$

I conjecture that for any $\beta>0$, \begin{align} \sigma(\beta) > \Phi(x_{\beta}) \quad\text{where}\quad x_{\beta} = \beta \phi(x_{\beta}). \end{align}

  • $\sigma$ denotes the sigmoid function defined by $\sigma(x) = \frac{1}{1+e^{-x}}$.
  • $\Phi$ and $\phi$ denote the standard normal cdf and pdf, respectively.

The conjecture seems to be true according to my numerical analysis, but I am unsure about a formal proof.

I would appreciate it if you would share your thoughts or suggestions. Thank you.

$\endgroup$
3
  • $\begingroup$ $x_{\beta} = \beta \phi(x_{\beta})$ looks like a circular definition $\endgroup$
    – Henry
    Commented Mar 28 at 2:14
  • 1
    $\begingroup$ Thank you for your comment. The definition is not circular, however. The equation $x = \beta \phi(x)$ has a unique solution, $x_{\beta} > 0$ for all $\beta > 0$. $\endgroup$
    – Pierre
    Commented Mar 28 at 3:11
  • $\begingroup$ It is true for $\beta >0$. From your $x_{\beta} = \beta \phi(x_{\beta})$, consider $b(x)=\frac{x}{\phi(x)}$ which is a bijection $\mathbb R \to \mathbb R$ which preserves signs so has an inverse. You can say $\beta=b(x_\beta)$, i.e. $x_\beta=b^{-1}(\beta)$. Your assertion becomes that for any $x_\beta>0$ you have $\dfrac{1}{1+e^{-x_\beta/\phi(x_\beta)}} >\Phi(x_\beta)$, which is true - to prove it you might want to separate into small and large $x_\beta$ and find simple functions between the LHS and RHS. $\endgroup$
    – Henry
    Commented Mar 28 at 12:53

1 Answer 1

0
$\begingroup$

We rephrase the problem as follows.

Problem. Prove that, for all $\beta > 0$, $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \int_{-\infty}^{u} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t$$ where $u$ is a real number satisfying $$u = \beta \cdot \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-u^2/2}.$$ (Note: Clearly, such $u$ exists (unique) and is positive.)

$\phantom{2}$

Proof.

We split into two cases.

Case 1. $\beta \ge 3$

Let $f(u) := u - \beta \cdot \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-u^2/2}$. We have $f'(u) > 0$. We have $$f(\sqrt{2\ln \beta}) = \sqrt{2\ln \beta} - \frac{1}{\sqrt{2\pi}} \ge \sqrt{2\ln 3} - \frac{1}{\sqrt{2\pi}} > 0 = f(u)$$ which results in $\sqrt{2\ln \beta} \ge u$. It suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \int_{-\infty}^{\sqrt{2\ln \beta}} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{1}$$

We have, for all $u>0$ $$1 - \frac{u^2 - 1}{u^3\sqrt{2\pi}}\,\mathrm{e}^{-u^2/2}\ge \int_{-\infty}^{u} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{2}$$ (Proof: Let $g(u) := \mathrm{LHS} - \mathrm{RHS}$. We have $g'(u) = - \frac{3}{u^4\sqrt{2\pi}}\mathrm{e}^{-u^2/2}< 0$ for all $u > 0$. Also, $g(\infty) = 0$. Thus, $g(u)\ge 0$ for all $u > 0$.)

From (1) and (2), it suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > 1 - \frac{2\ln \beta - 1}{\sqrt{2\ln \beta}^3\sqrt{2\pi}}\cdot \frac{1}{\beta}. \tag{3}$$

We have \begin{align*} &\mathrm{e}^{-\beta} = \frac{1}{\mathrm{e}^3\mathrm{e}^{\beta-3}} \le \frac{1}{\mathrm{e}^3[1 + (\beta - 3) + \frac12(\beta - 3)^2]}, \tag{4}\\[6pt] &\frac{2\ln \beta - 1}{\sqrt{2\ln \beta}^3\sqrt{2\pi}} = \frac{1 - 1/(2\ln\beta)}{\sqrt{2\ln \beta}\sqrt{2\pi}} \ge \frac{1 - 1/(2\ln 3)}{\sqrt{2\ln \beta}\sqrt{2\pi}},\tag{5}\\[6pt] &\ln \beta = \ln 3 + \ln (1 + (\beta - 3)/3) \le \ln 3 + (\beta - 3)/3. \tag{6} \end{align*}

From (3)-(6), it suffices to prove that $$\frac{1}{1 + \frac{1}{\mathrm{e}^3[1 + (\beta - 3) + \frac12(\beta - 3)^2]}} > 1 - \frac{1 - 1/(2\ln 3)}{\sqrt{2\ln 3 + 2(\beta - 3)/3}\cdot \sqrt{2\pi}}\cdot \frac{1}{\beta},$$ or $$\frac{1 - 1/(2\ln 3)}{\sqrt{2\ln 3 + 2(\beta - 3)/3}\cdot \sqrt{2\pi}}\cdot \frac{1}{\beta} > \frac{2}{\mathrm{e}^3(\beta^2 - 4\beta + 5) + 2},$$ or $$\frac{(1 - 1/(2\ln 3))^2}{[2\ln 3 + 2(\beta - 3)/3]\cdot 2\pi}\cdot \frac{1}{\beta^2} > \frac{4}{[\mathrm{e}^3(\beta^2 - 4\beta + 5) + 2]^2},$$ or (using $\mathrm{e}^3 > 20$ and $2\ln 3 > 2 + 1/6$ and $2\ln 3 < 2 + 1/5$ and $2\pi < 6 + 1/3$) $$\frac{(1 - 1/(2 + 1/6))^2}{[(2 + 1/5) + 2(\beta - 3)/3]\cdot (6 + 1/3)}\cdot \frac{1}{\beta^2} > \frac{4}{[20(\beta^2 - 4\beta + 5) + 2]^2},$$ or $${\frac {1}{3211}}\,{\frac {220500\,{\beta}^{4}-1796110\,{\beta}^{3}+5767467\,{ \beta}^{2}-8996400\,\beta+5735205}{ \left( 3+10\,\beta \right) {\beta}^{2} \left( 10\, {\beta}^{2}-40\,\beta+51 \right) ^{2}}} > 0$$ which is true.

$\phantom{2}$

Case 2. $0 < \beta < 3$

Let $f(u) := u - \beta \cdot \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-u^2/2}$. We have $f'(u) > 0$. Using $\sqrt{2\pi} > 5/2$ and $\mathrm{e}^{-2\beta^2/25} = \frac{1}{\mathrm{e}^{2\beta^2/25}} \le \frac{1}{1+ 2\beta^2/25}$, we have $$f(2\beta/5) = \frac{2\beta}{5} - \frac{\beta \mathrm{e}^{-2\beta^2/25}}{\sqrt{2\pi}} \ge \frac{2\beta}{5} - \frac{ \beta \cdot \frac{2}{5}}{1+ 2\beta^2/25} = \frac{4\beta^3}{10\beta^2 + 125} > 0 = f(u)$$ which results in $2\beta/5 \ge u$. It suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{7}$$

(i) If $11/4 \le \beta < 3$, we have $$\frac{1}{1 + \mathrm{e}^{-\beta}} - \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t \ge \frac{1}{1 + \mathrm{e}^{-11/4}} - \int_{-\infty}^{2\cdot 3/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t > 0. $$

(ii) If $0 < \beta < 11/4$, it suffices to prove that $$\frac{1}{1 + \mathrm{e}^{-\beta}} > \frac{2\beta}{5\sqrt{2\pi}} + \frac12 > \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t. \tag{8}$$

Let $F(\beta) := \frac{1}{1 + \mathrm{e}^{-\beta}} - \frac{2\beta}{5\sqrt{2\pi}} - \frac12$. We have $F''(\beta) := - \frac{\mathrm{e}^\beta(\mathrm{e}^\beta - 1)}{(1 + \mathrm{e}^\beta)^3} > 0$. Also, $F(0) = 0$ and $F(11/4) > 0 $. Thus, $F(\beta) > 0$ on $(0, 11/4)$.

Let $G(\beta) := \frac{2\beta}{5\sqrt{2\pi}} + \frac12 - \int_{-\infty}^{2\beta/5} \frac{1}{\sqrt{2\pi}}\mathrm{e}^{-t^2/2}\, \mathrm{d} t$. We have $G'(\beta) = \frac{2(1 - \mathrm{e}^{-2\beta^2/25})}{5\sqrt{2\pi}} > 0$. Also, $G(0) = 0$. Thus, $G(\beta) > 0$ on $(0, 11/4)$.

Thus, (8) is true.

We are done.

$\endgroup$
2
  • $\begingroup$ Thank you very much. It is perfect. May I ask (from my curiosity) how you came up with the series of ingenious inequalities? $\endgroup$
    – Pierre
    Commented Mar 29 at 23:18
  • $\begingroup$ @Pierre For example, LHS of (2) is a result from the asymptotic expansion of the RHS of (2). (8) is to find a linear segment separating the two curves. We can do it with the help of Maple - a computer algebra system. $\endgroup$
    – River Li
    Commented Mar 30 at 0:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .