9
$\begingroup$

In Pitman's Probability, the tail sum formula for expectation is introduced for a nonnegative (0,1,...) discrete random variable $X$:

$$E(X) = \sum_{i=0}^\infty P(X > i).$$

  1. I wonder if there is a similar formula for nonnegative continuous random variable $X$:

    $$E(X) = \int_0^\infty P(X > x) dx?$$

    If no, are there some conditions for it to hold? And how can it be proved?

    Here is my thought:

    If the cdf $F$ of $X$ is bijective, then $X=F^{-1}(U)$ for some random variable $U$ uniformly distributed over $[0,1)$. So $$E(X) = \int_0^1 F^{-1}(u) du.$$

    To prove the tail sum formula, it suffices to prove $$\int_0^1 F^{-1}(u) du = \int_0^\infty P(X > x) dx.$$ But I am stuck here.

    What's more, is the condition that the cdf $F$ of $X$ is bijective really necessary for tail sum formula to hold?

  2. Can tail sum formula be generalized to a random variable that is not necessarily nonnegative?

Thanks!

$\endgroup$
  • 1
    $\begingroup$ I've seen it written as a Riemann-Stieltjes integral $\int_0^\infty x\;dF(x)$, where $F(x) = \Pr(X \le x)$ is the cumulative distribution function. This works regardless of whether you have a continuous distribution, a discrete distribution, a singular distribution (which is neither continuous nor discrete nor a mixture of the two), or a mixture of any or all of the three kinds. Only for continuous distributions is this the same as $\int_0^\infty x f(x)\;dx$, where $f(x)=F'(x)$ is the probability density function. $\endgroup$ – Michael Hardy Sep 12 '11 at 1:26
  • $\begingroup$ @Michael: Thanks! The tail sum formula is another way to computer expectation defined as in your comment. $\endgroup$ – Tim Sep 12 '11 at 2:45
  • 1
    $\begingroup$ Similar question: stats.stackexchange.com/questions/18438/… $\endgroup$ – Henry Nov 26 '11 at 13:10
  • $\begingroup$ In case others stumble upon this question and are also using Pitman's textbook: the continuous case is covered in Exercise 9 of Section 4.5. $\endgroup$ – angryavian Aug 1 '16 at 23:45
10
$\begingroup$

This is Fubini theorem for nonnegative functions/sums. If $x$ is a nonnegative integer,

$$ x=\sum_{i=0}^{+\infty}[x\gt i]. $$

Likewise, if $x$ is a nonnegative real number,

$$ x=\int_{0}^{+\infty}[x\gt t]\mathrm dt,\qquad x=\int_{0}^{+\infty}[x\geqslant t]\mathrm dt. $$

Then one integrates both sides of the relevant identity with respect to the distribution $\mathrm P_X$ of $X$ and one uses Fubini theorem to change the order of the summation/integral and of the expectation.

For example, the second identity yields $$ \mathbb E(X)=\int_\Omega X\ \mathrm d\mathbb P=\int_\Omega \int_{0}^{+\infty}[X\gt t]\mathrm dt\ \mathrm d\mathbb P=\int_{0}^{+\infty}\int_\Omega [X\gt t]\mathrm d\mathbb P\ \mathrm dt $$ that is,

$$\mathbb E(X)=\int_{0}^{+\infty}\mathbb P(X\gt t)\ \mathrm dt. $$

Likewise, the first identity yields $$ \mathbb E(X)=\int_\Omega X\ \mathrm d\mathbb P=\int_\Omega\ \sum\limits_{i=0}^{+\infty}[X\gt i]\ \mathrm d\mathbb P=\sum\limits_{i=0}^{+\infty}\ \int_\Omega[X\gt i]\ \mathrm d\mathbb P $$ that is,

$$\mathbb E(X)=\sum\limits_{i=0}^{+\infty}\mathbb P(X\gt i). $$

$\endgroup$
  • $\begingroup$ What does the notation dP mean? I am not familiar with this definition of expectation. $\endgroup$ – Mong H. Ng Jan 27 at 2:31
  • $\begingroup$ @MongH.Ng Really? This is rather surprising. How do you define the expectation of a random variable then? $\endgroup$ – Did Jan 27 at 8:39
  • $\begingroup$ In my entire college career, I have only ever see E[X} = \int_{-\infty}^{\infty} x f(x) dx. I am guessing they are equivalent, but I am not sure how to parse this. $\endgroup$ – Mong H. Ng Jan 27 at 17:31
  • $\begingroup$ @MongH.Ng Well, so you cannot compute $E(X)$ when $X$ has no PDF? Anyway, start from your formula, note that $f(x)=-\frac d{dx}P(X\geqslant x)$ and $1=\frac d{dx}x$ hence, by an integration by parts, $$\int_0^\infty xf(x)dx=\left[-xP(X\geqslant x)\right]_0^\infty-\int_0^\infty1\cdot(-P(X\geqslant x))dx$$ hence you are done. $\endgroup$ – Did Jan 27 at 19:28
  • $\begingroup$ @MongH.Ng en.wikipedia.org/wiki/… $\endgroup$ – Did Jan 27 at 19:30
6
$\begingroup$

This is integration by parts. Take $X\geq 0$, and call its distribution function $F$. Let $g$ be an increasing differentiable function with $g(0)=0$.

$$\begin{align*} \mathbb{E}[g(X)]&=\int_0^\infty g(t) dF(t) \\ &= \int_0^\infty -g(t) d(1-F(t)) \\ &= [-g(t)(1-F(t))]^\infty_0 - \int_0^\infty 1-F(t)d(-g(t)) \\ &= \int_0^\infty g'(t)\mathbb{P}[X>t]dt \end{align*}$$

This reduces to what you want when $g(X)=X$.

One way we could compute $\mathbb{E}[X]$ for general $X$ would be to compute $\mathbb{E}[X^+]$ and $\mathbb{E}[X^-]$ in this way and then take the difference, where $X^+=\max(X,0)$ and $X^-=\max(-X,0)$.

Justification of integration by parts: The integration by parts works when $g(X)$ is integrable, since, by dominated convergence, $$\begin{align} \limsup_x g(x)\mathbb{P}[X>x] &\le \limsup_x\mathbb{E}[g(X)\mathbf{1}(X>x)]\\ &= \mathbb{E}[\limsup_x g(X)\mathbf{1}(X>x)]\\ &= 0. \end{align}$$

$\endgroup$
  • $\begingroup$ Why is $g(+\infty)(1-F(+\infty))=0$? Ok, $1-F(+\infty)=0$ but what if $g(t)\to +\infty$? Does $g$ have to be bounded? Or does $1-F(t)$ converge so fast to $0$, that this works for any $g$? Thanks. $\endgroup$ – Jimmy R. Jan 4 '16 at 14:24
  • $\begingroup$ @Stef I've edited the answer in response to your question. $\endgroup$ – Ben Derrett Jan 4 '16 at 20:42
  • $\begingroup$ Ok, thanks. Yes, of course, if $g$ is integrable this works fine. $\endgroup$ – Jimmy R. Jan 4 '16 at 21:09
4
$\begingroup$

First we prove that for $X \geq 0$ $$EX= \int_{0}^{\infty}P(X > t)dt.$$ We apply Fubini's theorem. Our product measure will be a product of the distribution of $X$ and Lebesgue measure. So $$\int_{0}^{\infty}P(X>t)dt = \int_0^{\infty}\left( \int_{t}^{\infty} \nu_X(ds)\right)dt=\int_{\mathbb{R}}\left( \int_{0}^{\infty}\textbf{1}_{[0,\infty)}(t)\textbf{1}_{(t,\infty)}(s)\nu_X(ds)\right)dt=$$ $$=\int_{0}^{\infty}\left(\int_0^s dt \right) \nu_X(ds)= \int_{0}^{\infty}s \nu_X(ds)=\int_{\mathbb{R}}s\nu_X(ds)=EX$$

But, you want to have $EX=\int_{0}^{\infty}P(X \geq t)dt.$

Applying what we have already proved, it is enough to show that $\int_{0}^{\infty}P(X=t)dt=0$.

$\endgroup$
  • 1
    $\begingroup$ We can also show that for $X \geq 0$ and for an increasing differentiable function $g$, such that $g(0)=0$ we have $$E(g(X))= \int_{0}^{\infty} g^{\prime}(t) P(X>t) dt.$$ $\endgroup$ – Edvin Goey Sep 12 '11 at 1:16
  • $\begingroup$ Thanks! Don't worry about $EX=\int_{0}^{\infty}P(X \geq t)dt$. I misunderstood Pitman's book for the discrete case. $\endgroup$ – Tim Sep 12 '11 at 1:31
  • $\begingroup$ No problem, but $EX= \int_{0}^{\infty}P(X \geq t)dt$. I proved the main part for this, to get $\geq$ instead of $>$ it is sufficient to show that $\int_0^{\infty}P(X=t)dt=0$ which is obvious. $\endgroup$ – Edvin Goey Sep 12 '11 at 1:35
  • 2
    $\begingroup$ Edvin: This is obvious when using the right approach but maybe not so obvious to every MSE reader, hence I wonder why you skip the proof of this part... $\endgroup$ – Did Nov 25 '11 at 17:28
1
$\begingroup$

@Did gave a great answer, and while I'm thinking about it I want to record a similar answer using slightly different notation in the case where the cumulative density function $F$ is differentiable.

In this case, \begin{align} \mathbb E(X) &= \int_0^\infty x F'(x) \, dx \\ &= \int_0^\infty \int_0^\infty [x \geq t] \, dt \,F'(x) \, dx \\ &= \int_0^\infty \int_0^\infty [x \geq t] F'(x) \, dt \, dx \\ &= \int_0^\infty \int_0^\infty [x \geq t] F'(x) \, dx \, dt \quad \text{(Fubini's theorem)}\\ &= \int_0^\infty \int_t^\infty F'(x) \, dx \, dt \\ &= \int_0^\infty 1 - F(t) \, dt. \end{align}

(The quantity $[x \geq t]$ is equal to $1$ if $x \geq t$ and $0$ otherwise.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.