Let's say we have two independent random variables, $x_1$ and $x_2$, both have a probability mass function $X$ defined as

$$X(n) = \begin{cases} 2^{-m} & \text{if $n=2^m$ for $1 \le m \in \mathbb Z$} \\ 0 & \text{otherwise} \end{cases}$$

So $X(2)=\frac 12$, $X(4) = \frac 14$, $X(8) = \frac 18$, and so on (this is the distribution used in the St. Petersburg paradox).

The expected value is undefined for both $x_1$ and $x_2$. My question is, does $E[x_1-x_2]$ exist? I know that if it exists, it must be $0$, by symmetry. I have a feeling it does equal $0$, but I'm not sure.

In general, if two random variables have the same probability distribution, when will the expected value of their difference be $0$? (I think there would be counter examples for this.)


In probability theory, we use Lebesgue integration to define the expected value. In this sense your expected value is not defined, even as $\pm \infty$, because it is a $\infty - \infty$ form. That is, in Lebesgue integration you define $\int f = \int f^+ - \int f^-$, where $f^+$ is the positive part and $f^-$ is the negative part. Here both are $\infty$, so you get $\infty - \infty$ which we decline to define.

You could define integration through a notion like the Cauchy principal value; in this case the "symmetry argument" you mentioned would say that you do indeed get zero. This is often a bad idea because if you take the limit with broken symmetry (say summing from $-n$ to $2n$ and then sending $n \to \infty$) then you get a different limit.

  • $\begingroup$ @PyRulez 1. It doesn't matter that it's discrete, Lebesgue integration uses measures, which can be discrete. 2. In Lebesgue integration the definition of $\int f d \mu$ is $\int f^+ d \mu - \int f^- d \mu$, where $f^+$ is the positive part and $f^-$ is the negative part. So add up the portion of the sum where $x_1-x_2$ is positive and the portion where it's negative, and see that they are both infinite. $\endgroup$ – Ian Jan 24 '16 at 17:06
  • $\begingroup$ @PyRulez In limits, $\infty - \infty$ is indeterminate. That is, if $x_n \to \infty$ and $y_n \to \infty$ then $\lim_{n \to \infty} (x_n-y_n)$ can exist. In Lebesgue integration, the two are separated by definition, so you get $\infty - \infty$ as a number, not as a form. We decline to define this quantity. By doing so, we get that the Lebesgue integral is always absolutely convergent (unlike the improper Riemann integral). $\endgroup$ – Ian Jan 24 '16 at 17:07
  • $\begingroup$ Why the downvote here? $\endgroup$ – copper.hat Jan 24 '16 at 17:09
  • $\begingroup$ @Ian do you have a proof or link? $\endgroup$ – PyRulez Jan 24 '16 at 17:09
  • 1
    $\begingroup$ The linearity of expectation relies on both expectations being finite. Surely you would agree that the expectation of $x_1-x_1$ is 0, for example. $\endgroup$ – Kevin O'Bryant Jan 24 '16 at 17:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.