23
$\begingroup$

How do we go about deriving the values of mean and variance of a Gaussian Random Variable $X$ given its probability density function ?

$\endgroup$
3
  • 3
    $\begingroup$ A couple of integrations. $\endgroup$ – André Nicolas Oct 8 '13 at 0:34
  • $\begingroup$ Funny thing is that given the density of Gaussian you do not need even an integration to find the mean and variance! $\endgroup$ – Arash Oct 8 '13 at 0:40
  • $\begingroup$ @MichaelHardy, You are right, I missed the term "deriving" at the question. $\endgroup$ – Arash Oct 8 '13 at 1:04
45
$\begingroup$

UPDATE 21-03-2017
A much faster way is to differentiate both sides of

$$\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx=1$$ with respect to the two parameters $\mu$ and $\sigma^2$ (RHS will then be zero).


The Gaussian pdf is defined as $$f_X(x) =\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}$$

where $\mu$ and $\sigma$ are two parameters, with $\sigma >0$. By definition of the mean we have $$E(X) = \int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$ which using integral properties can be written as

$$E(X) = \int_{-\infty}^{\infty}(x+\mu)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$=\int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \;+\; \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \qquad [1]$$

For the first integral, call it $I_1$ we have using additivity

$$I_1 = \int_{-\infty}^0x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$ Swapping the integration limits in the first we have

$$I_1 = -\int_{0}^{-\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

and using again integral properties we have

$$I_1 = \int_{0}^{\infty}(-x)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(-x)^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$\Rightarrow I_1 = -\int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx = 0\qquad [2]$$

So we have that

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$

Multiply by $\sigma \sqrt2$ to obtain

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sqrt{\pi}}e^{-x^2} dx = \mu\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx$$

...the last term because the integrand is an even function.

Now $$\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx = \lim_{t\rightarrow \infty}\frac{2}{\sqrt{\pi}}\int_{0}^{t}e^{-x^2} dx = \lim_{t\rightarrow \infty} \text{erf}(t) = 1$$

where "erf" is the error function. So we end up with $$E(X) = \mu$$ i.e. that the parameter $\mu$ is the mean of the distribution.

VARIANCE
We have

$$\text {Var}(X) = \int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$

Applying the same tricks as before we have

$$\int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx = \int_{-\infty}^{\infty}x^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$

$$=\sigma \sqrt2\int_{-\infty}^{\infty}(\sigma \sqrt2x)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(\sigma \sqrt2x)^2}{2\sigma^2}\right\}dx = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}x^2e^{-x^2}dx$$

Define $t=x^2\Rightarrow x= \sqrt t$ and $dt = 2xdx = 2\sqrt tdx \Rightarrow dx = (2\sqrt t)^{-1}dt$. Substituting

$$V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}(\sqrt t)^2(2\sqrt t)^{-1}e^{-t}dt = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \int_{0}^{\infty}t^{\frac 32 -1}e^{-t}dt= \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \Gamma\left(\frac 32\right)$$

$$\Rightarrow V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \frac {\sqrt \pi}{2} = \sigma^2$$

where $\Gamma()$ is the Gamma function. So the parameter $\sigma$ is the square-root of the variance, i.e. the standard deviation.

$\endgroup$
5
  • $\begingroup$ Hey, I'm trying to wrap my head around your derivation but I don't understand the step before [1] where you say "Using integral properties, this can be written as...". Could you provide some information on that step? Thx $\endgroup$ – Sandi Nov 2 '18 at 10:04
  • 1
    $\begingroup$ @Sandi Let $$\int_a^b xf(x-\lambda) dx$$ Define $v \equiv x-\lambda \implies x = v+\lambda \implies dv = dx$, and also $x = a \implies v = a- \lambda , x=b \implies v=b - \lambda$. Make the substitution to get $$\int_a^b xf(x-\lambda) dx = \int_{a-\lambda}^{b-\lambda} (v+\lambda)f(v)dv$$ Note that $\infty -\lambda = \infty$. Don't forget that under the integral the variable of integration is a dummy, it does not carry any specific properties. So it is customary (although admittedly confusing) to keep using the same symbol (i.e. keep $x$ as I did in my post, instead of using $v$). $\endgroup$ – Alecos Papadopoulos Nov 2 '18 at 10:29
  • $\begingroup$ When you multiply by $\sigma \sqrt{2}$ how do you cancel out the denominator of $e$? $\endgroup$ – Char Nov 3 '18 at 22:37
  • 1
    $\begingroup$ @Char Define $w = x/\sigma \sqrt{2}$ and make the substitution. $\endgroup$ – Alecos Papadopoulos Nov 4 '18 at 2:06
  • 1
    $\begingroup$ @AlecosPapadopoulos: Thanks, I added modified versions of your proofs here and here. $\endgroup$ – Joram Soch Jan 15 '20 at 10:17
11
$\begingroup$

If $x\mapsto f(x)$ is the density function of a random variable $X$ with expected value $0$ and variance $1$, then $x\mapsto \frac1\sigma f\left(\frac{x-\mu}{\sigma}\right)$ is the density function of $\mu+\sigma X$, and thus of a random variable with expected value $\mu$ and variance $\sigma^2$.

That can be shown by thinking about the substitution $u = \dfrac{x-\mu}{\sigma}$ and $du=\dfrac{dx}\sigma$.

Therefore the problem reduces to this: How do we show that $$ x\mapsto \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right) $$ is the density of a distribution with expectation $0$ and variance $1$?

If you know that the expectation of a distribution with density $f$ is $\int_{-\infty}^\infty xf(x)\,dx$, then you know that you need to find $$ \int_{-\infty}^\infty x \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx. $$

Since this is the integral of an odd function over an interval that is symmetric about $0$, it must be equal to $0$ unless the positive and negative parts both diverge to $\infty$. The positive part is a constant times $$ \int_0^\infty x \exp\left(\frac{-x^2}{2}\right)\,dx = \int_0^\infty \exp(u)\,du, $$ where $u=-x^2/2$ so $du=x\,dx$. Clearly this is finite, and the negative part can be treated the same way.

The variance is \begin{align} & \int_{-\infty}^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2 \int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty \Big(x\Big)\Big(x\exp\left(\frac{-x^2}{2}\right)\,dx\Big) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int x\,dv \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left( xv-\int v\,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left(\left[-x\exp\left(\frac{-x^2}{2}\right)\right]_0^\infty -\int_0^\infty -\exp\left(\frac{-x^2}{2}\right) \,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int_0^\infty \exp\left(\frac{-x^2}{2}\right) \,dx. \end{align}

You know that this is equal to $1$ if you know how the normalizing constant in the density function was found.

$\endgroup$
2
  • $\begingroup$ Here's an amusing additional exercise. Consider the function $(x\in\mathbb R^n)\mapsto$ $c \cdot\exp\left( \frac{-1}{2} (x-\mu)^\top A^{-1} (x-\mu) \right)$, where $c$ is chosen to make this a density function and $A$ is an $n\times n$ positive definite matrix. Prove that $\mu\in\mathbb R^n$ is the expected value and the variance is $A=\mathbb E((X-\mu)(X-\mu)^\top) \in \mathbb R^{n\times n}$. ${}\qquad{}$ $\endgroup$ – Michael Hardy Oct 8 '13 at 0:59
  • $\begingroup$ Thanks for the explanation, there's a sign confusion in the line "where $u=-x^2/2$ so $du=x\,dx$", should be $du=-x\,dx$ $\endgroup$ – blz Oct 21 '18 at 7:28
0
$\begingroup$

Another way that might be easier to conceptualize:

As defined earlier, 𝐸(𝑋)= $\int_{-∞}^∞ xf(x)dx$

To make this easier to type out, I will call $\mu$ 'm' and $\sigma$ 's'

f(x)= $\frac{1}{\sqrt{(2πs^2)}}$exp{ $\frac{-(x-m)^2}{(\sqrt{2s^2}}$}. So, putting in the full function for f(x) will yield

E(x)= $\int_{-∞}^∞ x \frac{1}{\sqrt{(2πs^2)}}$exp{ $\frac{-(x-m)^2}{(\sqrt{2s^2} }dx$. Pretty gross to look at.

Looking at the exponent that e is being raised to, it seems like an ideal candidate for a u-subsitution. I will set u= $\frac {x-m}{s\sqrt2}$. That means that du= $\frac{dx}{s\sqrt2}$, or in other words dx= du•$s\sqrt2$.

Let's simplify the integral as much as possible using what we now know. Recalling the term $\frac{1}{\sqrt{(2πs^2)}}$, we shall rewrite it as $\frac{1}{s\sqrt{(2π)}}$.

E(x)= $\int_{-∞}^∞ x \frac{1}{s\sqrt{(2π)}}$ exp {$-u^2$}du•$s\sqrt2$.

The $s\sqrt2$'s from the first and last terms will cancel, leaving a constant of $\frac{1}{\sqrtπ}$. Let's put this in front of the integral for now to make this look a little better.

E(x)= $\frac{1}{\sqrtπ}$$\int_{-∞}^∞ x $ exp {$-u^2$}du.

The derivative of $\int_{-∞}^∞ $ exp {$-u^2$}du is in the form of a Gaussian integral. I'm not going to pretend that I can easily derive it (multivariable calc???) but it is widely known that the value of this integral is $\sqrtπ$.

We can use this knowledge to perform integration by parts to determine the value of the integral. I like to use u and v for this, but this u is not the same as the u from earlier. I hope this still makes sense.

$\int{udv}$=uv-$\int{ vdu}$

Set u=x. This means u'= 1dx. In terms of dx from earlier, u'= $s\sqrt2$ du. v'= $e^{(-u^2)}$du, and so by definition v= $\sqrtπ$.

uv= x$\sqrtπ$ and vdu=$\sqrtπ s\sqrt2 du$.

The overall integral is now equal to $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $\int_{-∞}^∞ \sqrtπ s\sqrt2 du$] evaluated from ∞ to -∞.

Antideriving $\int vdu$ produces $s\sqrt2π$u. Recall our earlier value of u, u= $\frac {x-m}{s\sqrt2}$.

E(x)= $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $s\sqrt2π$$\frac{x-m}{s\sqrt2}$] or $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $\sqrtπ$(x-m)] from ∞ to -∞.

Within the brackets, the terms being evaluated from ∞ to -∞ are:

x$\sqrtπ$-x$\sqrtπ$ + m$\sqrtπ$= m$\sqrtπ$.

E(x)= $\frac{1}{\sqrtπ}$ • m$\sqrtπ$ = m, or the mean, just like the definition of the expected value says.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.