# how to derive the mean and variance of a Gaussian Random variable?

How do we go about deriving the values of mean and variance of a Gaussian Random Variable $X$ given its probability density function ?

• A couple of integrations. – André Nicolas Oct 8 '13 at 0:34
• Funny thing is that given the density of Gaussian you do not need even an integration to find the mean and variance! – Arash Oct 8 '13 at 0:40
• @MichaelHardy, You are right, I missed the term "deriving" at the question. – Arash Oct 8 '13 at 1:04

UPDATE 21-03-2017
A much faster way is to differentiate both sides of

$$\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx=1$$ with respect to the two parameters $\mu$ and $\sigma^2$ (RHS will then be zero).

The Gaussian pdf is defined as $$f_X(x) =\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}$$

where $\mu$ and $\sigma$ are two parameters, with $\sigma >0$. By definition of the mean we have $$E(X) = \int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$ which using integral properties can be written as

$$E(X) = \int_{-\infty}^{\infty}(x+\mu)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$=\int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \;+\; \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \qquad $$

For the first integral, call it $I_1$ we have using additivity

$$I_1 = \int_{-\infty}^0x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$ Swapping the integration limits in the first we have

$$I_1 = -\int_{0}^{-\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

and using again integral properties we have

$$I_1 = \int_{0}^{\infty}(-x)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(-x)^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$\Rightarrow I_1 = -\int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx = 0\qquad $$

So we have that

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

Multiply by $\sigma \sqrt2$ to obtain

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sqrt{\pi}}e^{-x^2} dx = \mu\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx$$

...the last term because the integrand is an even function.

Now $$\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx = \lim_{t\rightarrow \infty}\frac{2}{\sqrt{\pi}}\int_{0}^{t}e^{-x^2} dx = \lim_{t\rightarrow \infty} \text{erf}(t) = 1$$

where "erf" is the error function. So we end up with $$E(X) = \mu$$ i.e. that the parameter $\mu$ is the mean of the distribution.

VARIANCE
We have

$$\text {Var}(X) = \int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$

Applying the same tricks as before we have

$$\int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx = \int_{-\infty}^{\infty}x^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$=\sigma \sqrt2\int_{-\infty}^{\infty}(\sigma \sqrt2x)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(\sigma \sqrt2x)^2}{2\sigma^2}\right\}dx = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}x^2e^{-x^2}dx$$

Define $t=x^2\Rightarrow x= \sqrt t$ and $dt = 2xdx = 2\sqrt tdx \Rightarrow dx = (2\sqrt t)^{-1}dt$. Substituting

$$V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}(\sqrt t)^2(2\sqrt t)^{-1}e^{-t}dt = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \int_{0}^{\infty}t^{\frac 32 -1}e^{-t}dt= \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \Gamma\left(\frac 32\right)$$

$$\Rightarrow V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \frac {\sqrt \pi}{2} = \sigma^2$$

where $\Gamma()$ is the Gamma function. So the parameter $\sigma$ is the square-root of the variance, i.e. the standard deviation.

• Hey, I'm trying to wrap my head around your derivation but I don't understand the step before  where you say "Using integral properties, this can be written as...". Could you provide some information on that step? Thx – Sandi Nov 2 '18 at 10:04
• @Sandi Let $$\int_a^b xf(x-\lambda) dx$$ Define $v \equiv x-\lambda \implies x = v+\lambda \implies dv = dx$, and also $x = a \implies v = a- \lambda , x=b \implies v=b - \lambda$. Make the substitution to get $$\int_a^b xf(x-\lambda) dx = \int_{a-\lambda}^{b-\lambda} (v+\lambda)f(v)dv$$ Note that $\infty -\lambda = \infty$. Don't forget that under the integral the variable of integration is a dummy, it does not carry any specific properties. So it is customary (although admittedly confusing) to keep using the same symbol (i.e. keep $x$ as I did in my post, instead of using $v$). – Alecos Papadopoulos Nov 2 '18 at 10:29
• When you multiply by $\sigma \sqrt{2}$ how do you cancel out the denominator of $e$? – Char Nov 3 '18 at 22:37
• @Char Define $w = x/\sigma \sqrt{2}$ and make the substitution. – Alecos Papadopoulos Nov 4 '18 at 2:06
• @AlecosPapadopoulos: Thanks, I added modified versions of your proofs here and here. – Joram Soch Jan 15 at 10:17

If $$x\mapsto f(x)$$ is the density function of a random variable $$X$$ with expected value $$0$$ and variance $$1$$, then $$x\mapsto \frac1\sigma f\left(\frac{x-\mu}{\sigma}\right)$$ is the density function of $$\mu+\sigma X$$, and thus of a random variable with expected value $$\mu$$ and variance $$\sigma^2$$.

That can be shown by thinking about the substitution $$u = \dfrac{x-\mu}{\sigma}$$ and $$du=\dfrac{dx}\sigma$$.

Therefore the problem reduces to this: How do we show that $$x\mapsto \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)$$ is the density of a distribution with expectation $$0$$ and variance $$1$$?

If you know that the expectation of a distribution with density $$f$$ is $$\int_{-\infty}^\infty xf(x)\,dx$$, then you know that you need to find $$\int_{-\infty}^\infty x \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx.$$

Since this is the integral of an odd function over an interval that is symmetric about $$0$$, it must be equal to $$0$$ unless the positive and negative parts both diverge to $$\infty$$. The positive part is a constant times $$\int_0^\infty x \exp\left(\frac{-x^2}{2}\right)\,dx = \int_0^\infty \exp(u)\,du,$$ where $$u=-x^2/2$$ so $$du=x\,dx$$. Clearly this is finite, and the negative part can be treated the same way.

The variance is \begin{align} & \int_{-\infty}^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2 \int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty \Big(x\Big)\Big(x\exp\left(\frac{-x^2}{2}\right)\,dx\Big) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int x\,dv \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left( xv-\int v\,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left(\left[-x\exp\left(\frac{-x^2}{2}\right)\right]_0^\infty -\int_0^\infty -\exp\left(\frac{-x^2}{2}\right) \,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int_0^\infty \exp\left(\frac{-x^2}{2}\right) \,dx. \end{align}

You know that this is equal to $$1$$ if you know how the normalizing constant in the density function was found.

• Here's an amusing additional exercise. Consider the function $(x\in\mathbb R^n)\mapsto$ $c \cdot\exp\left( \frac{-1}{2} (x-\mu)^\top A^{-1} (x-\mu) \right)$, where $c$ is chosen to make this a density function and $A$ is an $n\times n$ positive definite matrix. Prove that $\mu\in\mathbb R^n$ is the expected value and the variance is $A=\mathbb E((X-\mu)(X-\mu)^\top) \in \mathbb R^{n\times n}$. ${}\qquad{}$ – Michael Hardy Oct 8 '13 at 0:59
• Thanks for the explanation, there's a sign confusion in the line "where $u=-x^2/2$ so $du=x\,dx$", should be $du=-x\,dx$ – blz Oct 21 '18 at 7:28