2
$\begingroup$

Does anyone know how to analytically solve the below expression?

$$\int_{-\infty}^\infty p(x) \times \sigma(ax + b) \, dx $$

where $ p(x) = N(0, 1) = \frac{1}{\sqrt{2\pi}}e^{\frac{-x^2}{2}}$ and $\sigma(z) = \frac{1}{1+e^{-z}}$.

I know that the integration of standard Gaussian is $\pi$. Moreover, the integration of sigmoid alone is,

$$\int \sigma(x) \, df = \int \frac{e^x}{1+e^x} \, dx = x + c$$

But I cant figure out how to integrate the first equation), when they are multiplied together.

Can we use integration by part and solve above equation?

Any help will be appreciated. Thanks

Edit 1:

Sorry about the complicated notations. If we simplify the above expression, then we get following equation. $$ \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty \frac{e^{-\frac{x^2}{2}}}{1+e^{-x}} dx $$

$\endgroup$

marked as duplicate by Did, Sil, Namaste integration Aug 11 '18 at 12:36

This question was marked as an exact duplicate of an existing question.

  • $\begingroup$ What is $p(x)$ ? Is it the probability density function ? $\endgroup$ – MysteryGuy Aug 2 '18 at 12:11
  • $\begingroup$ yes. $p(x) = \frac{1}{\sqrt{2\pi}}e^{\frac{-x^2}{2}}$ $\endgroup$ – Nadheesh Aug 2 '18 at 12:17
  • $\begingroup$ If $x$ is between $\pm\infty,$ as opposed to $x$ having an imaginary part, then $\ln|e^x|$ is redundant and one can just write $\ln e^x,$ and $\ln e^x$ is the same as $x,$ so $\ln e^x + c = x + c. \qquad$ $\endgroup$ – Michael Hardy Aug 2 '18 at 12:26
  • $\begingroup$ @MichaelHardy thanks for the suggestion. edited the post accordingly. However, that is not what I'm interested in. I want to solve the first expression. $\endgroup$ – Nadheesh Aug 2 '18 at 12:33
  • $\begingroup$ Sorry I’m not familiar with this topic...but what is $N(0,1)$? $\endgroup$ – Szeto Aug 2 '18 at 12:35
4
$\begingroup$

\begin{align} \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty e^{-\frac{x^2}{2}}\frac{e^x}{1+e^{x}} dx&=\frac{1}{\sqrt{2\pi}} \int_{0}^\infty e^{-\frac{x^2}{2}}\frac{e^x}{1+e^{x}} dx+\frac{1}{\sqrt{2\pi}} \int_{-\infty}^0 e^{-\frac{x^2}{2}}\frac{e^x}{1+e^{x}} dx\\ &=\frac{1}{\sqrt{2\pi}} \int_{0}^\infty e^{-\frac{x^2}{2}}\frac{e^x}{1+e^{x}} dx+\frac{1}{\sqrt{2\pi}} \int_0^{\infty} e^{-\frac{x^2}{2}}\frac{e^{-x}}{1+e^{-x}} dx\\ &=\frac{1}{\sqrt{2\pi}} \int_{0}^\infty e^{-\frac{x^2}{2}}\frac{e^x}{1+e^{x}} dx+\frac{1}{\sqrt{2\pi}} \int_0^{\infty} e^{-\frac{x^2}{2}}\frac{1}{1+e^{x}} dx\\ &=\frac{1}{\sqrt{2\pi}} \int_{0}^\infty e^{-\frac{x^2}{2}} dx\\ &=\frac12 \end{align}

$\endgroup$
  • $\begingroup$ Thanks for all the work, I would have never thought of this myself. How can we solve the expression if the sigmoid function include some other parameters as show below? $$ \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty \frac{e^{-\frac{x^2}{2}}}{1+e^{-(ax+b)}} dx $$ $\endgroup$ – Nadheesh Aug 3 '18 at 1:54
  • $\begingroup$ Going from the second to the first line, why can we change the integration bounds and $e^x$ $\endgroup$ – badatmath Aug 27 '18 at 9:01
  • $\begingroup$ I used $x \to -x$ transformation. $\endgroup$ – Math-fun Aug 27 '18 at 9:15
2
$\begingroup$

$$ \varphi(x) = \frac 1 {\sqrt{2\pi}} e^{-x^2/2} $$ $$ \sigma (x) = \frac 1 {1+e^{-x}} $$ Some simple algebra shows that $$ \sigma(-x) = 1 - \sigma(x) $$ and it is even easier to show that $$ \varphi(-x) = \varphi(x), $$ so we have $$ \int_{-\infty}^0 \varphi(x)\,dx = \int_0^\infty \varphi(x)\,dx = \frac 1 2. $$

Therefore \begin{align} \int_{-\infty}^0 \sigma(x) \varphi(x) \, dx & = \int_0^{+\infty} (1-\sigma(x)) \varphi(x)\, dx \\[10pt] & = \int_0^{+\infty} \varphi(x)\,dx - \int_0^{+\infty} \sigma(x)\varphi(x)\,dx \end{align} Therefore $$ \int_{-\infty}^{+\infty} \sigma(x)\varphi(x)\,dx = \int_0^{+\infty} \varphi(x)\, dx = \frac 1 2. $$

$\endgroup$
  • 1
    $\begingroup$ Thanks for the solution. This seems pretty simple yet, I could not have think of without your help. However, if we have some other variables within the sigmoid function, then how can we solve the same expression. I tried substituting z = ax+b, then I could not get rid of the sigmoid. $$ \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty \frac{e^{-\frac{x^2}{2}}}{1+e^{-(ax+b)}} dx $$ $\endgroup$ – Nadheesh Aug 3 '18 at 1:57
  • $\begingroup$ do you think if we can solve this expression when the sigmoid function is changed as shown above. $\endgroup$ – Nadheesh Aug 4 '18 at 2:48

Not the answer you're looking for? Browse other questions tagged or ask your own question.