27
$\begingroup$

How do we go about deriving the values of mean and variance of a Gaussian Random Variable $X$ given its probability density function ?

$\endgroup$
3
  • 4
    $\begingroup$ A couple of integrations. $\endgroup$ Commented Oct 8, 2013 at 0:34
  • $\begingroup$ Funny thing is that given the density of Gaussian you do not need even an integration to find the mean and variance! $\endgroup$
    – Arash
    Commented Oct 8, 2013 at 0:40
  • $\begingroup$ @MichaelHardy, You are right, I missed the term "deriving" at the question. $\endgroup$
    – Arash
    Commented Oct 8, 2013 at 1:04

4 Answers 4

52
$\begingroup$

UPDATE 21-03-2017
A much faster way is to differentiate both sides of

$$\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx=1$$ with respect to the two parameters $\mu$ and $\sigma^2$ (RHS will then be zero).


The Gaussian pdf is defined as $$f_X(x) =\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}$$

where $\mu$ and $\sigma$ are two parameters, with $\sigma >0$. By definition of the mean we have $$E(X) = \int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$ which using integral properties can be written as

$$E(X) = \int_{-\infty}^{\infty}(x+\mu)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$=\int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \;+\; \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \qquad [1]$$

The first integral, call it $I_1$ equals zero, because we integrate an odd function (the product of an odd times an even function gives an odd function), over an interval centered at zero. Being pedantic, we have using additivity

$$I_1 = \int_{-\infty}^0x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$ Swapping the integration limits in the first we have

$$I_1 = -\int_{0}^{-\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

and using again integral properties we have

$$I_1 = \int_{0}^{\infty}(-x)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(-x)^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$\Rightarrow I_1 = -\int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx = 0\qquad [2]$$

So we have that

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$

Multiply by $\sigma \sqrt2$ to obtain

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sqrt{\pi}}e^{-x^2} dx = \mu\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx$$

...the last term because the integrand is an even function.

Now $$\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx = \lim_{t\rightarrow \infty}\frac{2}{\sqrt{\pi}}\int_{0}^{t}e^{-x^2} dx = \lim_{t\rightarrow \infty} \text{erf}(t) = 1$$

where "erf" is the error function. So we end up with $$E(X) = \mu$$ i.e. that the parameter $\mu$ is the mean of the distribution.

VARIANCE
We have

$$\text {Var}(X) = \int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$

Applying the same tricks as before we have

$$\int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx = \int_{-\infty}^{\infty}x^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$

$$=\sigma \sqrt2\int_{-\infty}^{\infty}(\sigma \sqrt2x)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(\sigma \sqrt2x)^2}{2\sigma^2}\right\}dx = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}x^2e^{-x^2}dx$$

Define $t=x^2\Rightarrow x= \sqrt t$ and $dt = 2xdx = 2\sqrt tdx \Rightarrow dx = (2\sqrt t)^{-1}dt$. Substituting

$$V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}(\sqrt t)^2(2\sqrt t)^{-1}e^{-t}dt = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \int_{0}^{\infty}t^{\frac 32 -1}e^{-t}dt= \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \Gamma\left(\frac 32\right)$$

$$\Rightarrow V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \frac {\sqrt \pi}{2} = \sigma^2$$

where $\Gamma()$ is the Gamma function. So the parameter $\sigma$ is the square-root of the variance, i.e. the standard deviation.

$\endgroup$
8
  • $\begingroup$ Hey, I'm trying to wrap my head around your derivation but I don't understand the step before [1] where you say "Using integral properties, this can be written as...". Could you provide some information on that step? Thx $\endgroup$
    – Sandi
    Commented Nov 2, 2018 at 10:04
  • 1
    $\begingroup$ @Sandi Let $$\int_a^b xf(x-\lambda) dx$$ Define $v \equiv x-\lambda \implies x = v+\lambda \implies dv = dx$, and also $x = a \implies v = a- \lambda , x=b \implies v=b - \lambda$. Make the substitution to get $$\int_a^b xf(x-\lambda) dx = \int_{a-\lambda}^{b-\lambda} (v+\lambda)f(v)dv$$ Note that $\infty -\lambda = \infty$. Don't forget that under the integral the variable of integration is a dummy, it does not carry any specific properties. So it is customary (although admittedly confusing) to keep using the same symbol (i.e. keep $x$ as I did in my post, instead of using $v$). $\endgroup$ Commented Nov 2, 2018 at 10:29
  • 1
    $\begingroup$ @Char Define $w = x/\sigma \sqrt{2}$ and make the substitution. $\endgroup$ Commented Nov 4, 2018 at 2:06
  • 1
    $\begingroup$ Just for I1, I would rather simply say that the function is odd and you integrate it on an interval centered around 0 $\endgroup$
    – MysteryGuy
    Commented Sep 10, 2023 at 15:53
  • 1
    $\begingroup$ @MysteryGuy You are right, just that sometimes it is useful to show all the intermediate tedious steps. But I added this as an explanation of what I am showing. $\endgroup$ Commented Sep 10, 2023 at 23:14
12
$\begingroup$

If $x\mapsto f(x)$ is the density function of a random variable $X$ with expected value $0$ and variance $1$, then $x\mapsto \frac1\sigma f\left(\frac{x-\mu}{\sigma}\right)$ is the density function of $\mu+\sigma X$, and thus of a random variable with expected value $\mu$ and variance $\sigma^2$.

That can be shown by thinking about the substitution $u = \dfrac{x-\mu}{\sigma}$ and $du=\dfrac{dx}\sigma$.

Therefore the problem reduces to this: How do we show that $$ x\mapsto \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right) $$ is the density of a distribution with expectation $0$ and variance $1$?

If you know that the expectation of a distribution with density $f$ is $\int_{-\infty}^\infty xf(x)\,dx$, then you know that you need to find $$ \int_{-\infty}^\infty x \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx. $$

Since this is the integral of an odd function over an interval that is symmetric about $0$, it must be equal to $0$ unless the positive and negative parts both diverge to $\infty$. The positive part is a constant times $$ \int_0^\infty x \exp\left(\frac{-x^2}{2}\right)\,dx = \int_0^\infty \exp(u)\,du, $$ where $u=-x^2/2$ so $du=x\,dx$. Clearly this is finite, and the negative part can be treated the same way.

The variance is \begin{align} & \int_{-\infty}^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2 \int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty \Big(x\Big)\Big(x\exp\left(\frac{-x^2}{2}\right)\,dx\Big) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int x\,dv \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left( xv-\int v\,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left(\left[-x\exp\left(\frac{-x^2}{2}\right)\right]_0^\infty -\int_0^\infty -\exp\left(\frac{-x^2}{2}\right) \,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int_0^\infty \exp\left(\frac{-x^2}{2}\right) \,dx. \end{align}

You know that this is equal to $1$ if you know how the normalizing constant in the density function was found.

$\endgroup$
2
  • $\begingroup$ Here's an amusing additional exercise. Consider the function $(x\in\mathbb R^n)\mapsto$ $c \cdot\exp\left( \frac{-1}{2} (x-\mu)^\top A^{-1} (x-\mu) \right)$, where $c$ is chosen to make this a density function and $A$ is an $n\times n$ positive definite matrix. Prove that $\mu\in\mathbb R^n$ is the expected value and the variance is $A=\mathbb E((X-\mu)(X-\mu)^\top) \in \mathbb R^{n\times n}$. ${}\qquad{}$ $\endgroup$ Commented Oct 8, 2013 at 0:59
  • $\begingroup$ Thanks for the explanation, there's a sign confusion in the line "where $u=-x^2/2$ so $du=x\,dx$", should be $du=-x\,dx$ $\endgroup$
    – blz
    Commented Oct 21, 2018 at 7:28
3
$\begingroup$

$$E(X)=E(x - \mu) + \mu$$ So we're done by showing that $E(x - \mu)=0$

$$E(x- \mu)= \int_{ -\infty }^\infty \frac{(x-\mu)}{\sqrt{2\pi}\sigma}\cdot e^{-\frac {(x-\mu)^2}{2\sigma^2}}\,dx$$

Using the subsitiution $z=\frac{x-\mu}{\sigma}$ we get

$$E(x- \mu)= \frac {\sigma}{\sqrt{2\pi}} \int_{ -\infty }^\infty z \cdot e^{z^2/2}\,dz$$ The function $f(z) = z\cdot e^{z^2/2}$ is obviously odd. Which makes $$\lim_{a \to \infty }\int_{ -a }^a z \cdot e^{z^2/2}\,dz=0 $$ and thus we're left with $$E(X)= \mu$$

$\endgroup$
0
$\begingroup$

Another way that might be easier to conceptualize:

As defined earlier, 𝐸(𝑋)= $\int_{-∞}^∞ xf(x)dx$

To make this easier to type out, I will call $\mu$ 'm' and $\sigma$ 's'

f(x)= $\frac{1}{\sqrt{(2πs^2)}}$exp{ $\frac{-(x-m)^2}{(\sqrt{2s^2}}$}. So, putting in the full function for f(x) will yield

E(x)= $\int_{-∞}^∞ x \frac{1}{\sqrt{(2πs^2)}}$exp{ $\frac{-(x-m)^2}{(\sqrt{2s^2} }dx$. Pretty gross to look at.

Looking at the exponent that e is being raised to, it seems like an ideal candidate for a u-subsitution. I will set u= $\frac {x-m}{s\sqrt2}$. That means that du= $\frac{dx}{s\sqrt2}$, or in other words dx= du•$s\sqrt2$.

Let's simplify the integral as much as possible using what we now know. Recalling the term $\frac{1}{\sqrt{(2πs^2)}}$, we shall rewrite it as $\frac{1}{s\sqrt{(2π)}}$.

E(x)= $\int_{-∞}^∞ x \frac{1}{s\sqrt{(2π)}}$ exp {$-u^2$}du•$s\sqrt2$.

The $s\sqrt2$'s from the first and last terms will cancel, leaving a constant of $\frac{1}{\sqrtπ}$. Let's put this in front of the integral for now to make this look a little better.

E(x)= $\frac{1}{\sqrtπ}$$\int_{-∞}^∞ x $ exp {$-u^2$}du.

The derivative of $\int_{-∞}^∞ $ exp {$-u^2$}du is in the form of a Gaussian integral. I'm not going to pretend that I can easily derive it (multivariable calc???) but it is widely known that the value of this integral is $\sqrtπ$.

We can use this knowledge to perform integration by parts to determine the value of the integral. I like to use u and v for this, but this u is not the same as the u from earlier. I hope this still makes sense.

$\int{udv}$=uv-$\int{ vdu}$

Set u=x. This means u'= 1dx. In terms of dx from earlier, u'= $s\sqrt2$ du. v'= $e^{(-u^2)}$du, and so by definition v= $\sqrtπ$.

uv= x$\sqrtπ$ and vdu=$\sqrtπ s\sqrt2 du$.

The overall integral is now equal to $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $\int_{-∞}^∞ \sqrtπ s\sqrt2 du$] evaluated from ∞ to -∞.

Antideriving $\int vdu$ produces $s\sqrt2π$u. Recall our earlier value of u, u= $\frac {x-m}{s\sqrt2}$.

E(x)= $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $s\sqrt2π$$\frac{x-m}{s\sqrt2}$] or $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $\sqrtπ$(x-m)] from ∞ to -∞.

Within the brackets, the terms being evaluated from ∞ to -∞ are:

x$\sqrtπ$-x$\sqrtπ$ + m$\sqrtπ$= m$\sqrtπ$.

E(x)= $\frac{1}{\sqrtπ}$ • m$\sqrtπ$ = m, or the mean, just like the definition of the expected value says.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .