16
$\begingroup$

How do we go about deriving the values of mean and variance of a Gaussian Random Variable $X$ given its probability density function ?

$\endgroup$
  • 2
    $\begingroup$ A couple of integrations. $\endgroup$ – André Nicolas Oct 8 '13 at 0:34
  • $\begingroup$ Funny thing is that given the density of Gaussian you do not need even an integration to find the mean and variance! $\endgroup$ – Arash Oct 8 '13 at 0:40
  • $\begingroup$ @MichaelHardy, You are right, I missed the term "deriving" at the question. $\endgroup$ – Arash Oct 8 '13 at 1:04
37
$\begingroup$

UPDATE 21-03-2017
A much faster way is to differentiate both sides of

$$\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx=1$$ with respect to the two parameters $\mu$ and $\sigma^2$ (RHS will then be zero).


The Gaussian pdf is defined as $$f_X(x) =\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}$$

where $\mu$ and $\sigma$ are two parameters, with $\sigma >0$. By definition of the mean we have $$E(X) = \int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$ which using integral properties can be written as

$$E(X) = \int_{-\infty}^{\infty}(x+\mu)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$=\int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \;+\; \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \qquad [1]$$

For the first integral, call it $I_1$ we have using additivity

$$I_1 = \int_{-\infty}^0x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$ Swapping the integration limits in the first we have

$$I_1 = -\int_{0}^{-\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

and using again integral properties we have

$$I_1 = \int_{0}^{\infty}(-x)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(-x)^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$\Rightarrow I_1 = -\int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx = 0\qquad [2]$$

So we have that

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$

Multiply by $\sigma \sqrt2$ to obtain

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sqrt{\pi}}e^{-x^2} dx = \mu\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx$$

...the last term because the integrand is an even function.

Now $$\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx = \lim_{t\rightarrow \infty}\frac{2}{\sqrt{\pi}}\int_{0}^{t}e^{-x^2} dx = \lim_{t\rightarrow \infty} \text{erf}(t) = 1$$

where "erf" is the error function. So we end up with $$E(X) = \mu$$ i.e. that the parameter $\mu$ is the mean of the distribution.

VARIANCE
We have

$$\text {Var}(X) = \int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$

Applying the same tricks as before we have

$$\int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx = \int_{-\infty}^{\infty}x^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$

$$=\sigma \sqrt2\int_{-\infty}^{\infty}(\sigma \sqrt2x)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(\sigma \sqrt2x)^2}{2\sigma^2}\right\}dx = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}x^2e^{-x^2}dx$$

Define $t=x^2\Rightarrow x= \sqrt t$ and $dt = 2xdx = 2\sqrt tdx \Rightarrow dx = (2\sqrt t)^{-1}dt$. Substituting

$$V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}(\sqrt t)^2(2\sqrt t)^{-1}e^{-t}dt = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \int_{0}^{\infty}t^{\frac 32 -1}e^{-t}dt= \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \Gamma\left(\frac 32\right)$$

$$\Rightarrow V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \frac {\sqrt \pi}{2} = \sigma^2$$

where $\Gamma()$ is the Gamma function. So the parameter $\sigma$ is the square-root of the variance, i.e. the standard deviation.

$\endgroup$
  • $\begingroup$ Hey, I'm trying to wrap my head around your derivation but I don't understand the step before [1] where you say "Using integral properties, this can be written as...". Could you provide some information on that step? Thx $\endgroup$ – Sandi Nov 2 '18 at 10:04
  • $\begingroup$ @Sandi Let $$\int_a^b xf(x-\lambda) dx$$ Define $v \equiv x-\lambda \implies x = v+\lambda \implies dv = dx$, and also $x = a \implies v = a- \lambda , x=b \implies v=b - \lambda$. Make the substitution to get $$\int_a^b xf(x-\lambda) dx = \int_{a-\lambda}^{b-\lambda} (v+\lambda)f(v)dv$$ Note that $\infty -\lambda = \infty$. Don't forget that under the integral the variable of integration is a dummy, it does not carry any specific properties. So it is customary (although admittedly confusing) to keep using the same symbol (i.e. keep $x$ as I did in my post, instead of using $v$). $\endgroup$ – Alecos Papadopoulos Nov 2 '18 at 10:29
  • $\begingroup$ When you multiply by $\sigma \sqrt{2}$ how do you cancel out the denominator of $e$? $\endgroup$ – Char Nov 3 '18 at 22:37
  • $\begingroup$ @Char Define $w = x/\sigma \sqrt{2}$ and make the substitution. $\endgroup$ – Alecos Papadopoulos Nov 4 '18 at 2:06
  • $\begingroup$ @AlecosPapadopoulos: Thanks, I added modified versions of your proofs here and here. $\endgroup$ – Joram Soch Jan 15 at 10:17
11
$\begingroup$

If $x\mapsto f(x)$ is the density function of a random variable $X$ with expected value $0$ and variance $1$, then $x\mapsto \frac1\sigma f\left(\frac{x-\mu}{\sigma}\right)$ is the density function of $\mu+\sigma X$, and thus of a random variable with expected value $\mu$ and variance $\sigma^2$.

That can be shown by thinking about the substitution $u = \dfrac{x-\mu}{\sigma}$ and $du=\dfrac{dx}\sigma$.

Therefore the problem reduces to this: How do we show that $$ x\mapsto \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right) $$ is the density of a distribution with expectation $0$ and variance $1$?

If you know that the expectation of a distribution with density $f$ is $\int_{-\infty}^\infty xf(x)\,dx$, then you know that you need to find $$ \int_{-\infty}^\infty x \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx. $$

Since this is the integral of an odd function over an interval that is symmetric about $0$, it must be equal to $0$ unless the positive and negative parts both diverge to $\infty$. The positive part is a constant times $$ \int_0^\infty x \exp\left(\frac{-x^2}{2}\right)\,dx = \int_0^\infty \exp(u)\,du, $$ where $u=-x^2/2$ so $du=x\,dx$. Clearly this is finite, and the negative part can be treated the same way.

The variance is \begin{align} & \int_{-\infty}^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2 \int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty \Big(x\Big)\Big(x\exp\left(\frac{-x^2}{2}\right)\,dx\Big) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int x\,dv \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left( xv-\int v\,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left(\left[-x\exp\left(\frac{-x^2}{2}\right)\right]_0^\infty -\int_0^\infty -\exp\left(\frac{-x^2}{2}\right) \,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int_0^\infty \exp\left(\frac{-x^2}{2}\right) \,dx. \end{align}

You know that this is equal to $1$ if you know how the normalizing constant in the density function was found.

$\endgroup$
  • $\begingroup$ Here's an amusing additional exercise. Consider the function $(x\in\mathbb R^n)\mapsto$ $c \cdot\exp\left( \frac{-1}{2} (x-\mu)^\top A^{-1} (x-\mu) \right)$, where $c$ is chosen to make this a density function and $A$ is an $n\times n$ positive definite matrix. Prove that $\mu\in\mathbb R^n$ is the expected value and the variance is $A=\mathbb E((X-\mu)(X-\mu)^\top) \in \mathbb R^{n\times n}$. ${}\qquad{}$ $\endgroup$ – Michael Hardy Oct 8 '13 at 0:59
  • $\begingroup$ Thanks for the explanation, there's a sign confusion in the line "where $u=-x^2/2$ so $du=x\,dx$", should be $du=-x\,dx$ $\endgroup$ – blz Oct 21 '18 at 7:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.