# Intuitive explanation of variance and moment in Probability

While I understand the intuition behind expectation, I don't really understand the meaning of variance and moment.

What is a good way to think of those two terms?

-
Why the edit, more than 4 years later? –  Did Dec 20 '14 at 13:05
@Did -- Sorry, was checking something and forgot edits bump up questions :( –  Pandora Dec 20 '14 at 13:13

There are many good answers possible, depending on how one's intuition works and one's experience with related things like averages, $L^p$ norms, etc. One answer that builds on your intuition of an expectation is to think of the higher central moments as weighted expectations. The central third moment, for example, is defined as the expectation of the third powers of deviations from the mean. Split a third power into two parts: one is the deviation from the mean (let's call it, after common statistical parlance, the "residual") and the other is the squared deviation from the mean (ergo, the squared residual). Treat the latter as a weight. As the residual gets larger in size, these weights get larger (in a quadratic fashion). The third central moment is the weighted "average" of the residuals. (I have to put "average" in quotes because, strictly speaking, to deserve being called an average one would require the weights themselves to average out to 1, but they don't usually do that.) Thus, the third central moment gives disproportionately greater emphasis to larger residuals compared to smaller ones. (It is possible for that emphasis to be so great that the third central moment doesn't even exist: if the probability doesn't decay rapidly enough with the size of the residual, the weights can swamp the probability, with the net effect of creating an infinitely great third central moment.)

You can understand, and even analyze, all other central moments in the same way, even fractional moments and absolute moments. It should be immediately clear, for instance, that the higher the moment, the more the weights are emphasized at extreme values: higher central moments measure (average) behavior of the probability distribution at greater distances from the mean.

-

Variance of a r.v. $X$ is $Var(X) = \mathbb{E}[(X-\mathbb{E}(X))^2]$. Variance is used as a measure for the dispersion of a random variable. If you are interested in how far a random variable is from what you expect; perhaps you would consider $\mathbb{E}(|X-\mathbb{E}(X)|)$ but see: http://stats.stackexchange.com/questions/118/standard-deviation-why-square-the-difference-instead-of-taking-the-absolute-val for why this we use the square instead.

For all except the first moment (i.e. the mean) the central moments (centred about the mean) are far more interesting.

As for the $r$th (central) moment of $X$, that is $\mathbb{E}[(X-\mathbb{E}(X))^r]$. The lower moments ($r<5$) are related to properties of the distribution:

$r=0$ is $1$

$r=1$ is $0$

$r=2$ is the variance which is a measure of the spread from the mean.

$r=3$ is related to skewness which is a measure of the asymmetry of the distribution. See: http://en.wikipedia.org/wiki/Skewness

$r=4$ is related to kurtois which is a measure of the 'peakedness' of a distribution. See: http://en.wikipedia.org/wiki/Kurtosis

For $r\geq5$ they become slightly harder to compute. The $5th$ central moment is a measure for the asymmetry of the distributions tails, but I would use skewness instead.

-
$r=3$ and $r=4$ are only related to skewness and kurtosis. For these moments to merit these names, they have to be normalized by an appropriate measure of spread (to wit, the standard deviation). –  whuber Sep 24 '10 at 18:32
Yes that is true. Thanks edited. –  alext87 Sep 24 '10 at 18:40
Thanks; +1 for that. You left off the punch line, though, and it might help to make it explicit by stating how one usually thinks of skewness (asymmetry) and kurtosis (peakedness or fat-tailedness). That would make it obvious that your answer really does address the request for "ways to think of" the moments. –  whuber Sep 24 '10 at 18:46
That was the wiki page's job. Though in hindsight it makes it a better answer. Edited +1. :) Whuber, out of curiousity, have you used the higher (central) moments for any practical work? –  alext87 Sep 24 '10 at 19:04
It depends on what you mean. If you mean for data analysis, the answer is a resounding no: even for quite large datasets, the higher moments are extremely sensitive to outliers and are not robust enough for drawing useful conclusions. Even the use of the third and fourth moments is awfully old-fashioned for that reason, but these moments can occasionally be useful measurements of the skewness and tailedness of an empirical distribution. Modern statistics tends to use extreme quantiles or order statistics to characterize the tails. –  whuber Sep 30 '10 at 4:38

The variance is the expected squared deviation between the random variable and its expectation.

It measures how much the random variable scatters or spreads around its expectation.

-
This is the definition, but I was hoping for some intuitive insight. –  Pandora Sep 24 '10 at 17:20
I was hoping that my second sentence would provide some intuition. It contains the common way of thinking of the variance. –  Rasmus Sep 24 '10 at 17:22
+1 for making the effort in the second sentence. Note, though, that the variance only indirectly measures spread (because it's not even in the same units of measurement): its square root, the standard deviation, more directly reflects spread. –  whuber Sep 24 '10 at 18:28

The localization and the dispersion probabilistic measures can be "seen" as the corresponding momentums of mechanical systems of "probabilistic masses".

The expectation has the following mechanical interpretation. Given that $F(x)$ is the "probalilistic mass" contained in the interval $0\le X\lt x$ (in one dimention), the mathematical expectation of the random variable $X$ is the static momentum with respect to the origin of the "probabilistic masses" system.

The variance is the mechanical analog of the inertia momentum of the "probabilistic masses" system with respect to the center of masses.

The variance of the random variable $X$ is the 2nd order momentum of $X-m$ where $m$ is the expectation of $X$, i.e. $m=E(X)=\displaystyle\int x dF(x)$ ($F(x)$ is the repartion function).

The $k$-order momentum is the expectation of $(X-m)^k$.

-