27
$\begingroup$

If $X$ is a discrete random variable that can take the values $x_1, x_2, \dots $ and with probability mass function $f_X$, then we define its mean by the number $$\sum x_i f_X(x_i) $$ (1) when the series above is absolutely convergent.

That's the definition of mean value of a discrete r.v. I've encountered in my books (Introduction to the Theory of Statistics by Mood A., Probability and Statistics by DeGroot M.).

I know that if a series is absolute convergent then it is convergent, but why do we need to ask for the series (1) to converge absolutely, instead of just asking it to converge? I'm taking my introductory courses of probabilty and so far I haven't found a situation that forces us restrict ourselves this way.

Any comments about the subject are appreciated.

$\endgroup$
4
  • 20
    $\begingroup$ I am not a probabilist but one guess is that when a series is not absolutely convergent, a lot of value is placed on the ordering of its terms, and in many applied contexts there is no natural ordering and you would not really care if something converged "by accident" of some chosen ordering. If you want unordered sums, you might want absolute convergence. This is only a guess. $\endgroup$ – leslie townes Apr 25 at 17:43
  • 1
    $\begingroup$ @leslietownes I like your explanation ! $\endgroup$ – Amelia Apr 25 at 17:49
  • 1
    $\begingroup$ I also suspect that the absolute convergence is needed for the Law of the unconscious statistician, as you have to re-order your sum in the process of the proof. $\endgroup$ – Gaetan Leclerc Apr 25 at 21:16
  • $\begingroup$ @leslietownes i see so much stuff in the answers here like about ordering. really? what about for random variables in general? i posted an answer. please let me know what you think. (to that extent of your probability knowledge) $\endgroup$ – BCLC Apr 26 at 10:47
36
$\begingroup$

It's because if the series is convergent but not absolutely convergent, you can rearrange the sum to get any value. Any good notion of "mean" or "expectation" should not depend on the ordering of the $x_i$'s.

For a more abstract reason, note that we define the expectation $E[X]$ of a random variable $X$ defined on a probability space $(\Omega, \mathcal{F}, P)$ as the Lebesgue integral $\int_{\Omega} X dP$. By definition of the Lebesgue integral, this is only well-defined if the integrand is absolutely integrable. If you learn more about measure theory, you will also learn why this definition makes sense. It is done to avoid strange situations like $\infty - \infty$ in the theory.

$\endgroup$
0
14
$\begingroup$

I think any explanation is going to make reference to the fact that without absolute convergence, the value of an infinite sum or an improper Riemann integral depends on the order in which the "pieces" are summed up. That alone may satisfy you, but it didn't fully satisfy me.

The more specific reason that satisfied me is "without absolute convergence the law of large numbers fails". The intuitive reason for this is that when you're taking sample averages, instead of integrating $x f(x)$ symmetrically, you're integrating it by Monte Carlo integration, literally picking locations randomly. As a consequence, if the integral for the mean only converges conditionally, then there is no guarantee that the sample averages have the same behavior along different sequences of samples, or even that the sample averages converge at all.

To see this on a computer, try running a program like this, which takes successive sample averages from the standard Cauchy distribution (which is symmetric about $0$, so its mean "would be zero if it made sense").

n=1e4;
x=pi*(rand(1,n)-1/2);
y=cumsum(tan(x))./(1:n);
plot(1:n,y)

This program as is will run in Matlab or Octave, but very similar programs can be run in other software with support for random numbers and plotting. What you see is quite dramatic jumps in the sample mean that occur when an entry of x gets too close to $\pi/2$ or $-\pi/2$, and which continue to occur even after thousands of samples have been drawn.

$\endgroup$
11
  • $\begingroup$ Thank you so much for your explanation ! I'll keep your comments in mind when I go further on my probability courses. $\endgroup$ – Amelia Apr 25 at 18:11
  • 1
    $\begingroup$ Of course, there are only finitely many floating point values, so if you churn through this for an insanely large number of data points, it should eventually converge... except that if you manage to get a big enough value, I imagine you could end up overflowing to inf or -inf, at which point it will saturate there or at nan if you're unlucky enough to hit both infinities. But I'm not sure if that can actually happen. $\endgroup$ – Kevin Apr 26 at 7:31
  • $\begingroup$ i see so much stuff in the answers here like about ordering. really? what about for random variables in general? i posted an answer. please let me know what you think. $\endgroup$ – BCLC Apr 26 at 10:47
  • 1
    $\begingroup$ @BCLC None of this is limited to the discrete setting; you can rearrange an integral on the line by essentially breaking it into pieces and reordering the pieces. Frankly I would say your answer is just reiterating the definitions, not really exposing any particular piece of intuition. $\endgroup$ – Ian Apr 26 at 11:38
  • 1
    $\begingroup$ @BCLC You are right that when only the positive or negative part has infinite expectation and not the other, one can sensibly define the expectation to be $\pm \infty$ as appropriate. $\endgroup$ – Ian Apr 26 at 11:43
2
$\begingroup$

There is a distinction to be made in math between features that are an essential part of a mathematical object, versus features that are used in a more arbitrary manner simply as labels so that we can discuss the objects.

For instance, if we're encoding demographic data, and one of the variables is race, to encode that data as an integer requires assigning numbers to the different races. What number a race gets is not an essential feature of the race, but simply an arbitrary number given to keep track of it. It would be rather weird if our calculation of the average value of some metric over the different races gave a different answer depending on how we labeled the races.

Similarly, the index that we assign different $x$ values is not an essential part of the data. Even if there's an "obvious" order, it's not the only ordering, and we would want our definition to not depend on picking the right order, or there being an obvious ordering. Consider the set of rational numbers. This set is countable, so it's possible to come up with a labeling of the rational numbers by integers, but there are many different ways of going about this. If two people used different methods for labeling the rational numbers and got different numbers for the mean, that would be a problem.

The fact that the formula has $\Sigma$ rather than $\Sigma_{i=0}^{\infty}$ emphasizes this: the mean is an attribute of the events and their probabilities, not of any particular indexing. We can define the $\Sigma$ operator without any reference to an indexing. The sum over a finite set can easily be defined, and for an infinite set $X$, we can define the limit of its sum as being the number $L$ such that for any positive $\epsilon$, $X$ can be split into a finite set $H$ and infinite set $T$ such that the sum over $H$ plus the sum over any finite subset of $T$ is within $\epsilon$ of $L$. i.e.

$\Sigma X = L$ if $\forall \epsilon >0,\exists H,T:H \cup T = X, H \cap T= \emptyset, |H|<\infty, \forall S \subset T, |S| < \infty \rightarrow |\Sigma X + \Sigma S-L|<\epsilon$

This definition (unless I've messed up somewhere) is equivalent to "uniform convergence"; the term "uniform" is a qualifier we have to add when defining sums with respect to an indexing to make that sum independent of the indexing.

$\endgroup$
1
  • $\begingroup$ Your definition is ultimately equivalent to absolute convergence, though the proof of that is not trivial. $\endgroup$ – Ian Apr 28 at 18:45
-4
$\begingroup$

It's a great question, but unfortunately, I'm going to give an advanced probability/a measure theory answer that answers for random variables in general (OP's original title is actually about random variables. It was actually me, who edited it to discrete). However, I really can't imagine an elementary answer that consists of something more than simply 'Because' (or in Tagalog/Filipino 'Kasi'. I guess this is more precise because Tagalog/Filipino also has a word 'dahil' for a different kind of 'because').

Elementary probability peeps (including OP, I guess) may ignore this, except possibly the 'last part'. Or maybe elementary probability peeps can try to skim this and try to get an idea or something instead of understanding the specific details:

Defining expectations of random variables (or Lebesgue integrals of measurable functions) is in steps defined in the standard machine:

  1. Expectation of indicator random variables
  2. Expectation of simple random variables
  3. Expectation of nonnegative random variables
  4. Expectation of general random variables.

After 1 and 2, we get to 3. Let $X$ be a random variable. Here, we can define $E[X^{+}]$ and $E[X^{-}]$ for the nonnegative random variables $X^{+}$ and $X^{-}$, where $X^{+}$ and $X^{-}$ are something like, resp, 'the nonnegative part of $X$' and the negative of the 'nonpositive part of $X$' s.t. $X = X^{+} - X^{-}$. Then, given $|X| = X^{+} + X^{-}$, we can define $E[|X|] := E[X^{+}] + E[X^{-}]$. (Wait I forgot. Maybe you don't define this. Maybe it's really just decomposed by linearity of expectation/integration...)

And then we get to 4:

$$E[X] := E[X^{+}] - E[X^{-}]$$

Now, this is defined for either

  1. $X$ that satisfies $E[|X|] < \infty$ --> Here, we say that $X$ is Lebesgue integrable.

  2. $X$ that satisfies $E[X^{+}] < \infty$ or $E[X^{-}] < \infty$ --> Here, we say that the Lebesgue integral of $X$ exists.

Here, Condition 1 implies ($E[X^{+}] < \infty$ and $E[X^{-}] < \infty$, which implies) Condition 2 but not conversely. I guess analogous to regular limits in elementary calculus it's like: $\lim x^2 = \infty$, so $\lim x^2$ 'exists' but is not 'existable'. Meanwhile, $\lim \frac1x = 0$, so $\lim \frac1x$ is 'existable'. So 'existable' is the same as the usual 'exists' in elementary calculus, but 'exists' here is like including $\pm \infty$.

Finally, the aforementioned 'last part':

  1. Anyway, I believe most texts on measure theory or advanced probability will consider specifically the Lebesgue integrable ones and not all of the Lesbesgue-integral-exists ones. On wiki, it says 'It turns out that this definition gives the desirable properties of the integral.' I assume this statement refers to Lebesgue integrable.

So now this begs the question as to why those texts refer to Lebesgue integrable random variables/measurable functions specifically. Well, there's probably (lol) some properties that Lebesgue integrable's satisfy that not all of the Lesbesgue-integral-exists ones do, but I figure it's just to avoid $\pm \infty$

  1. AlohaSine mentioned about $\infty - \infty$ cases in h answer, but I think it doesn't apply to the Lesbesgue-integral-exists ones:
  • For $E[X^{+}] < \infty$ and $E[X^{-}] = \infty$, $E[X] = - \infty$

  • For $E[X^{+}] = \infty$ and $E[X^{-}] < \infty$, $E[X] = \infty$

Wait actually I just realised that even for elementary level probability, we use this kinds of definitions eg St. Petersburg paradox: $E[X]$ doesn't really 'exist' in the sense that $E[|X|] = \infty$, but we do have that $E[X]$ '$=\infty$' under the idea that $E[|X|] = \infty$ because $E[X^{+}] = \infty$ while $E[X^{-}] < \infty$ (specifically $E[X^{-}] =0 $) as I believe.

  1. there's a lot of stuff in comments and answers about rearrangement in re conditional convergence. Not so familiar about this apart from wiki, but I think this (explicitly) explains only the case of discrete random variables and not for random variables in general (whenever their expectations exist...or are 'existable')
$\endgroup$
10
  • 2
    $\begingroup$ "this (explicitly) explains only the case of discrete random variables and not for random variables in general" - well, the question is only about discrete random variables, general random variables don't have a probability mass function. $\endgroup$ – Misha Lavrov Apr 26 at 12:18
  • $\begingroup$ @MishaLavrov as i said, i was the one who edited the question. are you the downvoter? $\endgroup$ – BCLC Apr 26 at 12:55
  • $\begingroup$ You edited the title, but the question already said that we were talking about a probability mass function (and had a summation). And called out "discrete random variable" in the body of the question. $\endgroup$ – Misha Lavrov Apr 26 at 13:26
  • 1
    $\begingroup$ It is not an artificial term. If I am working on a problem in discrete probability, then I am not interested in bringing Lebesgue measurability, and issues with nonmeasurable sets, into the picture. An answer like yours is too general for this question; it doesn't address the issue, because once you talk about measure spaces, the expected value isn't a sum that can converge conditionally or absolutely in the first place! It's not a good answer from the point of view of measure theory, either: it uses made-up words like "existable", and skips over the technicalities. So yes, I downvoted. $\endgroup$ – Misha Lavrov Apr 26 at 16:59
  • 2
    $\begingroup$ This answer is all over the place and doesn't answer the question. $\endgroup$ – qwr Apr 27 at 3:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.