When you apply a Gaussian probability densityfunction to Rectified Linear/Sigmoid function, you will get a rectified(*)/distorted Gaussian function.

(*)rectified Gaussian function is a mixture of delta and truncated gaussian as shown belown

I am wondering, what is the Expectation and Variance of this rectified/distorted Gaussian?

The Expectation of distorted Gaussian from Sigmoid is shown below.
Expected value of applying the sigmoid function to a normal distribution

How about Variance? or Expectation and Variance of rectified Gaussian from Rectified Linear?

Thank you.

  • 1
    $\begingroup$ It looks like what you are a calling a rectified Gaussian is $Y=\max[c,X]$, where $c$ is a constant and $X$ is Gaussian (say, mean $m$ and variance $\sigma^2>0$). You can find $E[Y]$, $E[Y^2]$, and $Var(Y)$ easily via the law of total expectation (condition on $X\geq c$ and $X<c$). $\endgroup$ – Michael Oct 11 '16 at 6:03
  • $\begingroup$ So for example rectifying at $0$ for a Normal distribution $N(\mu, \sigma^2)$ you get an expectation of $\mu\,\left(1-\Phi\left( -\dfrac{\mu}{\sigma}\right)\right) + \sigma\,\phi\left(-\dfrac{\mu}{\sigma}\right)$ $\endgroup$ – Henry Oct 11 '16 at 7:14
  • $\begingroup$ Thanks for advice, yes I was able to derive same equation for expectation. But Variance is very hard and I got a very long equation... too big to paste it here. $\endgroup$ – JimSD Oct 13 '16 at 10:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.