1
$\begingroup$

I'm wondering if there is a way to solve the following integral given that $$\mu, \sigma, w, b$$ are constants ?

$$\int_{-\infty}^{\infty} \frac{e^{\frac{-(x-\mu)^2}{2\sigma^2}}}{1+e^{-(wx+b)}} dx$$

UPDATE (If you are interested to know how I tried to solve the integral): The original problem is an optimization problem and I'm trying the solve the integrals inside it to be able to solve this optimization function until I got stuck at a specific step during solving the integral:

1.Solving the following optimization function: $$ argmax \prod_{i=1}^{n} \left [ y_i \left ( \int_{-\infty}^{\infty} \frac{e^\frac{-({x_i - \mu_i})^2}{2 \sigma_i^2}}{1+e^{-(wx_i + b)}} dx_i\right ) + (1-y_i) \left ( \int_{-\infty}^{\infty} \frac{e^\frac{-({x_i - \mu_i})^2}{2 \sigma_i^2} e^{-(wx_i+b)}}{1+e^{-(wx_i + b)}} dx_i\right ) \right ]$$

2.Simplifying the Expression to be: $$argmax \prod_{i=1}^{n} \left [ \left ( \int_{-\infty}^{\infty} \frac{e^\frac{-({x_i - \mu_i})^2 }{2 \sigma_i^2} e^{-(wx_i+b)}}{1+e^{-(wx_i + b)}} dx_i\right ) + y_i \left ( \int_{-\infty}^{\infty} \frac{e^\frac{-({x_i - \mu_i})^2}{2 \sigma_i^2} (1-e^{-(wx_i+b)})}{1+e^{-(wx_i + b)}} dx_i\right ) \right]$$

3.solving the FIRST integral: $$\int_{-\infty}^{\infty} \frac{e^\frac{-({x_i - \mu_i})^2 }{2 \sigma_i^2} e^{-(wx_i+b)}}{1+e^{-(wx_i + b)}} dx_i \Rightarrow \int_{-\infty}^{\infty} \frac{e^\frac{-({x - \mu})^2 }{2 \sigma^2} e^{-(wx+b)}}{1+e^{-(wx + b)}} dx$$

4.Substitution: Let $y = x - \mu \rightarrow dy = dx$: $$= \int_{-\infty}^{\infty} \frac{e^\frac{-({y})^2 }{2 \sigma^2} e^{-(wy + w\mu + b)}}{1+e^{-(wy + w\mu + b)}} dy$$

5.Substitution: Let $u = \frac{y}{\sqrt{2}\sigma} \rightarrow du = \frac{dy}{\sqrt{2}\sigma}$: $$ = \sqrt{2} \sigma \int_{-\infty}^{\infty} \frac{e^{-u^2} e^{-(\sqrt{2} \sigma w u + w \mu + b)}}{1+e^{-(\sqrt{2} \sigma w u + w \mu + b)}} du$$

6.Let $\alpha = \sqrt{2}\sigma w$ and $\beta = w\mu + b$: $$= \frac{\alpha}{w} \int_{-\infty}^{\infty} \frac{e^{-u^2} e^{-(\alpha u+ \beta)}}{1+e^{-(\alpha u+ \beta)}} du$$

7.Simplifying the integral: $$= \frac{\alpha}{w} \int_{-\infty}^{\infty} \frac{e^{-u^2} (1 + e^{-(\alpha u+ \beta)} -1 )}{1+e^{-(\alpha u+ \beta)}} du = \frac{\alpha}{w} \left [ \int_{-\infty}^{\infty} e^{-u^2} du - \int_{-\infty}^{\infty} \frac{e^{-u^2}}{1+e^{-(\alpha u+ \beta)}} du \right ] $$

8.Since $\int_{-\infty}^{\infty} e^{-u^2} du = \sqrt{\pi}~erf(\infty)$ and $erf(\infty) \approx 1$: $$= \frac{\alpha}{w} \left [ \sqrt{\pi}- \int_{-\infty}^{\infty} \frac{e^{-u^2}}{1+e^{-(\alpha u+ \beta)}} du \right ]$$

9.Simplifying and Solving the second integral in a similar way: $$\int_{-\infty}^{\infty} \frac{e^\frac{-({x_i - \mu_i})^2}{2 \sigma_i^2} (1-e^{-(wx_i+b)})}{1+e^{-(wx_i + b)}} dx_i \Rightarrow \int_{-\infty}^{\infty} \frac{e^\frac{-({x - \mu})^2}{2 \sigma^2} (1-e^{-(wx+b)})}{1+e^{-(wx + b)}} dx$$

10.Doing the same substitutions in steps (4,5,6): $$= \frac{\alpha}{w} \int_{-\infty}^{\infty} \frac{e^{-u^2} (1-e^{-(\alpha u+ \beta)})}{1+e^{-(\alpha u+ \beta)}} du$$

11.Simplifying the integral: $$= \frac{\alpha}{w} \left [ \int_{-\infty}^{\infty} \frac{e^{-u^2} }{1+e^{-(\alpha u+ \beta)}} du - \int_{-\infty}^{\infty} \frac{e^{-u^2} e^{-(\alpha u+ \beta)} }{1+e^{-(\alpha u+ \beta)}} du \right ]$$

12.The second integral of step (11) is similar to the integral in step (6) $\rightarrow$ by substituting result of step (8) in (11): $$= \frac{\alpha}{w} \left [ \int_{-\infty}^{\infty} \frac{e^{-u^2} }{1+e^{-(\alpha u+ \beta)}} du - \left ( \sqrt{\pi} - \int_{-\infty}^{\infty} \frac{e^{-u^2} }{1+e^{-(\alpha u+ \beta)}} du\right) \right ]$$

13.Simplifying the integral: $$= \frac{\alpha}{w} \left [ \int_{-\infty}^{\infty} \frac{2e^{-u^2} }{1+e^{-(\alpha u+ \beta)}} du - \sqrt{\pi} \right ]$$

So, I got stuck with the same integral in each of the two integrals that I want to evaluate.

$\endgroup$
6
  • 1
    $\begingroup$ By rescaling we can assume WLOG that $\mu=0$ and $\sigma=\frac{1}{\sqrt2}$ but I don’t see a simplification of the resulting integral $\endgroup$ Commented Dec 26, 2019 at 12:38
  • $\begingroup$ Welcome to MSE. Please read this text about how to ask a good question. $\endgroup$ Commented Dec 26, 2019 at 12:43
  • $\begingroup$ @JoséCarlosSantos I went through it quickly. Do you suggest any improvements to the question? The title maybe not good enough but I don't know how to be more specific. $\endgroup$
    – mmaher
    Commented Dec 26, 2019 at 13:17
  • 1
    $\begingroup$ You could show us what is it that you tried. $\endgroup$ Commented Dec 26, 2019 at 13:18
  • $\begingroup$ The Residue theorem may be useful here. Change the integral to be from $-R$ to $R$, then close the loop by a semi-circle. Use the Residue theorem to find the integral over the whole loop. Show that the integral along the semi-circle must go to $0$ as $R\to \infty$ (since it decays exponentially, while the arclength grows linearly). $\endgroup$ Commented Dec 26, 2019 at 22:48

1 Answer 1

2
$\begingroup$

Simplifying your integral $I$, we can reduce the number of constants, by the subsequent substitutions $x'=x-\mu$ and $u=x'/\sqrt{2}\sigma$. Define $\alpha=\sqrt{2}\sigma w$ and $\beta=\mu w+b$. Then, your integral, in units of $\sqrt{2}\sigma$, is $$I/\sqrt{2}\sigma=\int_{-\infty}^\infty \frac{e^{-u^2}}{1+e^{-(\alpha u+\beta)}}\,du$$ I haven't been able to crack the general case yet, but if $\beta=0$, then there is a delightful way to compute the resulting integral, which is then independent of $\alpha$. Indeed, first split this integral in two: $$\int_{-\infty}^\infty \frac{e^{-u^2}}{1+e^{-\alpha u}}\,du=\int_{-\infty}^0 \frac{e^{-u^2}}{1+e^{-\alpha u}}\,du + \int_{0}^\infty \frac{e^{-u^2}}{1+e^{-\alpha u}}\,du$$ The integral that is over the negative real line can be mapped back onto $[0,\infty)$ by the substitution $u\mapsto -u$, resulting in $$\int_0^\infty \frac{e^{-u^2}}{1+e^{\alpha u}}\,du=\int_0^\infty \frac{e^{-u^2}e^{-\alpha u}}{1+e^{-\alpha u}}\,du$$ Now, add both integrals back up, and marvel at the cancellation, $$I/\sqrt{2}\sigma=\int_0^\infty \frac{e^{-u^2}\left(1+e^{-\alpha u}\right)}{1+e^{-\alpha u}}\,du=\frac{\sqrt{\pi}}{2}.$$ Your integral is then $$\sqrt{2}\sigma \int_{-\infty}^\infty \frac{e^{-u^2}}{1+e^{-\alpha u}}\,du=\sigma\sqrt{\frac{\pi}{2}}.$$

$\endgroup$
2
  • $\begingroup$ Thanks Alot! This is really brilliant. However, in the substitution of $u\mapsto -u$ Is it supposed to be $\int_0^\infty \frac{e^{-u^2}}{1+e^{\alpha u}}\,du$ or $\int_0^\infty \frac{-e^{-u^2}}{1+e^{\alpha u}}\,du$ And, do you have any ideas even how is it possible to start cracking the general case? $\endgroup$
    – mmaher
    Commented Dec 29, 2019 at 9:40
  • $\begingroup$ The sign in my post should be correct, because although you gain a negative sign as a result of the measure (or differential) change, it is cancelled by the fact that $\int_\infty^0=-\int_0^\infty$. $\endgroup$
    – Diffusion
    Commented Dec 29, 2019 at 19:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .