Measure theoretic definition (e.g., Durrett, Brzezniak):

Given a probability space $(\Omega, \mathcal{F}, P)$ and a random variable $X: \Omega \to \mathbb{R}$ assumed to be integrable, $$\int_\Omega |X| dP < \infty$$ then the expectation of $X$ is defined by, $$E[X] = \int_\Omega X dP$$

Elementary probability definition (e.g., Sheldon Ross, Tsitsiklis),

The expectation of a random variable $X$ is

$$E[X] = \begin{cases} \sum\limits_{x:p(x)>0} x p(x) & X \text{ is discrete} \\ \int\limits_{-\infty}^\infty xf(x) dx & X \text{ is continuous} \end{cases}$$ where $p$ is the probability mass function and $f$ is the probability density function.

It seems to that

  1. the $dP$ symbol somehow magically turns into $p(x)$ and $f(x)dx$. I can't see the link here since I don't understand the symbol $dP$. Is it a function? Is it a constant? What is the $d$ doing there (that's not a part of the definition at all)?

  2. the $\int_\Omega$ symbol turns into $\sum$ or a $\int$ over the space of $x$ on a situational basis, which is again strange to me.

Can someone explain how the measure theoretic definition of the expectation turns into the ones that most (non-mathematician) students study in a course on probability?

  • $\begingroup$ This question may be of use to you. $\endgroup$ – FakeAnalyst56 Jul 7 at 1:28
  • $\begingroup$ To put simple, it is a form of 'change of variable'. $\endgroup$ – Sangchul Lee Jul 7 at 1:54
  • $\begingroup$ @Masacroso Hi! Do you know a source for this? In Durrett's book, it is written $\int_\Omega X dP = \int_s X(\omega) P(d\omega)$, which is not as good as what you have written, i.e., in terms of $F$, which is the cumulative probability distribution. I can't see how $P(d\omega) = dF(\omega)$ $\endgroup$ – Cauchy's Carrot Jul 7 at 3:09
  • $\begingroup$ @SquaringtheCircleisEasy if I remember correctly you can read it in the textbooks of Klenke or Lahiri. In my previous comment it must be "When $F$ is differentiable" instead. $\endgroup$ – Masacroso Jul 7 at 3:11
  • $\begingroup$ Instead of going from measure theoretic towards the "loosely stated" non-math definition, it might be easier to think about going the other way. The non-measure theoretic definition still should need to have the requirement that if the expectation is either infinite series or improper integral that they should be absolutely convergent. Then one realizes they dont capture everything, i.e., mixture, non-absolutely-continuous, etc. So, a better way instead of running through all cases is to define the structure related to the CDF, and then think of finding the expectation per @Masacroso comment. $\endgroup$ – Just_to_Answer Jul 7 at 3:11

If $X : \Omega \to \mathbb{R}$ is a random variable, it induces the probability measure on $\mathbb{R}$, often denoted as $P(X \in \cdot)$, defined as the map

$$ X \mapsto P(X \in A) $$

for $A \in \mathcal{B}(\mathbb{R})$. This is called the pushforward measure of $P$ by $X$. In this particular case, it is the same as the Stieltjes measure induced by $F_X(\cdot) = \mathbb{P}(X \leq \cdot)$, and so, we may interchangeably write

$$ \int_{\mathbb{R}} f(x) \, \mathrm{d}F_X(x) = \int_{\mathbb{R}} f(x) \, P(X \in \mathrm{d}x). $$

Under this setting, we have the following theorem. (You may also refer to Theorem 1.6.9 of Durrett 4.1Ed.)

Theorem (Change of variables) For any random variable $X : \Omega \to \mathbb{R}$ and for any Borel-measurable $f : \mathbb{R} \to [0, \infty]$, the following identity holds:

$$ \int_{\Omega} f(X(\omega)) \, P(\mathrm{d}\omega) = \int_{\mathbb{R}} f(x) \, P(X \in \mathrm{d}x) $$

Of course, the same conclusion continues to hold when $f$ is any $\mathbb{R}$-valued Borel-measurable function such that $|f(X)|$ is integrable. This easily follows by decomposing $f$ as $f_+ - f_-$, where $f_+$ (resp. $f_-$) is the positive part (resp. negative part) of $f$ and applying the above theorem to $f_{\pm}$ separately. In particular, we get

$$ E[X] = \int_{\mathbb{R}} x \, P(X \in \mathrm{d}x). $$

Here are some special cases:

  • $X$ has discrete distribution if and only if $P(X \in \cdot) = \sum_{i=1}^{\infty} p_X(x_i) \delta_{x_i}(\cdot)$, where $p_X$ is the PMF of $X$ and $\delta_x$ is the point mass at $x$. In such case, for $f \geq 0$,

    $$ E[X] = \int_{\mathbb{R}} x \, \sum_{i=1}^{\infty} p_X(x_i) \delta_{x_i}(\mathrm{d}x) = \sum_{i=1}^{\infty} \left( \int_{\mathbb{R}} x \delta_{x_i}(\mathrm{d}x) \right) p_X(x_i) = \sum_{i=1}^{\infty} x_i p_X(x_i) $$

  • $X$ has continuous distribution if and only if $P(X \in \mathrm{d}x) = f_X(x) \, \mathrm{d}x $, where $f_X$ is the PDF of $X$. In such case, for $f \geq 0$,

    $$ E[X] = \int_{\mathbb{R}} x f_X(x) \, \mathrm{d}x $$

  • 1
    $\begingroup$ it is the notation $P(X\in dx)$ common? It is the first time I see it? moreover: it is justified in some formal sense or just want to capture an idea? $\endgroup$ – Masacroso Jul 8 at 15:24
  • 1
    $\begingroup$ @Masacroso, At least it is not an idiosyncratic notation, and you can find usages throughout literature. On top of that, it is not a new notation, but rather a combination of two frequently used notations: (1) The map $x\mapsto f(x)$ can be abbreviated as $f(\cdot)$, and (2) if $\mu(\cdot)$ is a measure, then the integral of $f$ w.r.t. $\mu$ can be written as $\int f(x) \, \mu(\mathrm{d}x)$. $\endgroup$ – Sangchul Lee Jul 9 at 3:38
  • $\begingroup$ It might be useful to state that $\sum_{i=1}^{\infty} p_X(x_i) \delta_{x_i}(\cdot)$ is absolutely continuous w.r.t to the counting measure on $S = \{x_i, i\geq 1\}$, the density function being $x\mapsto p_X(x) 1_{S}(x)$. $\endgroup$ – Gabriel Romon Jul 10 at 8:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.