# how to derive the mean and variance of a Gaussian Random variable?

How do we go about deriving the values of mean and variance of a Gaussian Random Variable $X$ given its probability density function ?

• A couple of integrations. – André Nicolas Oct 8 '13 at 0:34
• Funny thing is that given the density of Gaussian you do not need even an integration to find the mean and variance! – Arash Oct 8 '13 at 0:40
• @MichaelHardy, You are right, I missed the term "deriving" at the question. – Arash Oct 8 '13 at 1:04

UPDATE 21-03-2017
A much faster way is to differentiate both sides of

$$\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx=1$$ with respect to the two parameters $\mu$ and $\sigma^2$ (RHS will then be zero).

The Gaussian pdf is defined as $$f_X(x) =\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}$$

where $\mu$ and $\sigma$ are two parameters, with $\sigma >0$. By definition of the mean we have $$E(X) = \int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$ which using integral properties can be written as

$$E(X) = \int_{-\infty}^{\infty}(x+\mu)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$=\int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \;+\; \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \qquad [1]$$

For the first integral, call it $I_1$ we have using additivity

$$I_1 = \int_{-\infty}^0x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$ Swapping the integration limits in the first we have

$$I_1 = -\int_{0}^{-\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

and using again integral properties we have

$$I_1 = \int_{0}^{\infty}(-x)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(-x)^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$\Rightarrow I_1 = -\int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx = 0\qquad [2]$$

So we have that

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

Multiply by $\sigma \sqrt2$ to obtain

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sqrt{\pi}}e^{-x^2} dx = \mu\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx$$

...the last term because the integrand is an even function.

Now $$\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx = \lim_{t\rightarrow \infty}\frac{2}{\sqrt{\pi}}\int_{0}^{t}e^{-x^2} dx = \lim_{t\rightarrow \infty} \text{erf}(t) = 1$$

where "erf" is the error function. So we end up with $$E(X) = \mu$$ i.e. that the parameter $\mu$ is the mean of the distribution.

VARIANCE
We have

$$\text {Var}(X) = \int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$

Applying the same tricks as before we have

$$\int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx = \int_{-\infty}^{\infty}x^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$=\sigma \sqrt2\int_{-\infty}^{\infty}(\sigma \sqrt2x)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(\sigma \sqrt2x)^2}{2\sigma^2}\right\}dx = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}x^2e^{-x^2}dx$$

Define $t=x^2\Rightarrow x= \sqrt t$ and $dt = 2xdx = 2\sqrt tdx \Rightarrow dx = (2\sqrt t)^{-1}dt$. Substituting

$$V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}(\sqrt t)^2(2\sqrt t)^{-1}e^{-t}dt = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \int_{0}^{\infty}t^{\frac 32 -1}e^{-t}dt= \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \Gamma\left(\frac 32\right)$$

$$\Rightarrow V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \frac {\sqrt \pi}{2} = \sigma^2$$

where $\Gamma()$ is the Gamma function. So the parameter $\sigma$ is the square-root of the variance, i.e. the standard deviation.

• Hey, I'm trying to wrap my head around your derivation but I don't understand the step before [1] where you say "Using integral properties, this can be written as...". Could you provide some information on that step? Thx – Sandi Nov 2 '18 at 10:04
• @Sandi Let $$\int_a^b xf(x-\lambda) dx$$ Define $v \equiv x-\lambda \implies x = v+\lambda \implies dv = dx$, and also $x = a \implies v = a- \lambda , x=b \implies v=b - \lambda$. Make the substitution to get $$\int_a^b xf(x-\lambda) dx = \int_{a-\lambda}^{b-\lambda} (v+\lambda)f(v)dv$$ Note that $\infty -\lambda = \infty$. Don't forget that under the integral the variable of integration is a dummy, it does not carry any specific properties. So it is customary (although admittedly confusing) to keep using the same symbol (i.e. keep $x$ as I did in my post, instead of using $v$). – Alecos Papadopoulos Nov 2 '18 at 10:29
• When you multiply by $\sigma \sqrt{2}$ how do you cancel out the denominator of $e$? – Char Nov 3 '18 at 22:37
• @Char Define $w = x/\sigma \sqrt{2}$ and make the substitution. – Alecos Papadopoulos Nov 4 '18 at 2:06
• @AlecosPapadopoulos: Thanks, I added modified versions of your proofs here and here. – Joram Soch Jan 15 '20 at 10:17

If $$x\mapsto f(x)$$ is the density function of a random variable $$X$$ with expected value $$0$$ and variance $$1$$, then $$x\mapsto \frac1\sigma f\left(\frac{x-\mu}{\sigma}\right)$$ is the density function of $$\mu+\sigma X$$, and thus of a random variable with expected value $$\mu$$ and variance $$\sigma^2$$.

That can be shown by thinking about the substitution $$u = \dfrac{x-\mu}{\sigma}$$ and $$du=\dfrac{dx}\sigma$$.

Therefore the problem reduces to this: How do we show that $$x\mapsto \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)$$ is the density of a distribution with expectation $$0$$ and variance $$1$$?

If you know that the expectation of a distribution with density $$f$$ is $$\int_{-\infty}^\infty xf(x)\,dx$$, then you know that you need to find $$\int_{-\infty}^\infty x \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx.$$

Since this is the integral of an odd function over an interval that is symmetric about $$0$$, it must be equal to $$0$$ unless the positive and negative parts both diverge to $$\infty$$. The positive part is a constant times $$\int_0^\infty x \exp\left(\frac{-x^2}{2}\right)\,dx = \int_0^\infty \exp(u)\,du,$$ where $$u=-x^2/2$$ so $$du=x\,dx$$. Clearly this is finite, and the negative part can be treated the same way.

The variance is \begin{align} & \int_{-\infty}^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2 \int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty \Big(x\Big)\Big(x\exp\left(\frac{-x^2}{2}\right)\,dx\Big) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int x\,dv \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left( xv-\int v\,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left(\left[-x\exp\left(\frac{-x^2}{2}\right)\right]_0^\infty -\int_0^\infty -\exp\left(\frac{-x^2}{2}\right) \,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int_0^\infty \exp\left(\frac{-x^2}{2}\right) \,dx. \end{align}

You know that this is equal to $$1$$ if you know how the normalizing constant in the density function was found.

• Here's an amusing additional exercise. Consider the function $(x\in\mathbb R^n)\mapsto$ $c \cdot\exp\left( \frac{-1}{2} (x-\mu)^\top A^{-1} (x-\mu) \right)$, where $c$ is chosen to make this a density function and $A$ is an $n\times n$ positive definite matrix. Prove that $\mu\in\mathbb R^n$ is the expected value and the variance is $A=\mathbb E((X-\mu)(X-\mu)^\top) \in \mathbb R^{n\times n}$. ${}\qquad{}$ – Michael Hardy Oct 8 '13 at 0:59
• Thanks for the explanation, there's a sign confusion in the line "where $u=-x^2/2$ so $du=x\,dx$", should be $du=-x\,dx$ – blz Oct 21 '18 at 7:28

Another way that might be easier to conceptualize:

As defined earlier, 𝐸(𝑋)= $$\int_{-∞}^∞ xf(x)dx$$

To make this easier to type out, I will call $$\mu$$ 'm' and $$\sigma$$ 's'

f(x)= $$\frac{1}{\sqrt{(2πs^2)}}$$exp{ $$\frac{-(x-m)^2}{(\sqrt{2s^2}}$$}. So, putting in the full function for f(x) will yield

E(x)= $$\int_{-∞}^∞ x \frac{1}{\sqrt{(2πs^2)}}$$exp{ $$\frac{-(x-m)^2}{(\sqrt{2s^2} }dx$$. Pretty gross to look at.

Looking at the exponent that e is being raised to, it seems like an ideal candidate for a u-subsitution. I will set u= $$\frac {x-m}{s\sqrt2}$$. That means that du= $$\frac{dx}{s\sqrt2}$$, or in other words dx= du•$$s\sqrt2$$.

Let's simplify the integral as much as possible using what we now know. Recalling the term $$\frac{1}{\sqrt{(2πs^2)}}$$, we shall rewrite it as $$\frac{1}{s\sqrt{(2π)}}$$.

E(x)= $$\int_{-∞}^∞ x \frac{1}{s\sqrt{(2π)}}$$ exp {$$-u^2$$}du•$$s\sqrt2$$.

The $$s\sqrt2$$'s from the first and last terms will cancel, leaving a constant of $$\frac{1}{\sqrtπ}$$. Let's put this in front of the integral for now to make this look a little better.

E(x)= $$\frac{1}{\sqrtπ}\int_{-∞}^∞ x$$ exp {$$-u^2$$}du.

The derivative of $$\int_{-∞}^∞$$ exp {$$-u^2$$}du is in the form of a Gaussian integral. I'm not going to pretend that I can easily derive it (multivariable calc???) but it is widely known that the value of this integral is $$\sqrtπ$$.

We can use this knowledge to perform integration by parts to determine the value of the integral. I like to use u and v for this, but this u is not the same as the u from earlier. I hope this still makes sense.

$$\int{udv}$$=uv-$$\int{ vdu}$$

Set u=x. This means u'= 1dx. In terms of dx from earlier, u'= $$s\sqrt2$$ du. v'= $$e^{(-u^2)}$$du, and so by definition v= $$\sqrtπ$$.

uv= x$$\sqrtπ$$ and vdu=$$\sqrtπ s\sqrt2 du$$.

The overall integral is now equal to $$\frac{1}{\sqrtπ}$$ [x$$\sqrtπ$$- $$\int_{-∞}^∞ \sqrtπ s\sqrt2 du$$] evaluated from ∞ to -∞.

Antideriving $$\int vdu$$ produces $$s\sqrt2π$$u. Recall our earlier value of u, u= $$\frac {x-m}{s\sqrt2}$$.

E(x)= $$\frac{1}{\sqrtπ}$$ [x$$\sqrtπ$$- $$s\sqrt2π\frac{x-m}{s\sqrt2}$$] or $$\frac{1}{\sqrtπ}$$ [x$$\sqrtπ$$- $$\sqrtπ$$(x-m)] from ∞ to -∞.

Within the brackets, the terms being evaluated from ∞ to -∞ are:

x$$\sqrtπ$$-x$$\sqrtπ$$ + m$$\sqrtπ$$= m$$\sqrtπ$$.

E(x)= $$\frac{1}{\sqrtπ}$$ • m$$\sqrtπ$$ = m, or the mean, just like the definition of the expected value says.