I would like to write the expected value of $c(x)$ where $x$ is sampled from a distribution $\gamma(x|m)$ and $m$ is sampled from another distribution $\omega(m)$. Here, for any fixed $m$, the distribution $\gamma(x | m)$ is a probability distribution supported on $\mathbb{R}$. Moreover, the distribution $\omega$ is a probability distribution over some (for simplicity) interval $M$.

By the law of total expectation, we have $$ \mathbb{E}[c(x)] = \int_{m \in M} \Big[\underbrace{\int_{x \in \mathbb{R}} c(x) \mathrm{d} \gamma(x | m)}_{\mathbb{E}[c(x) | f]} \Big] \mathrm{d}\omega(m).$$

I would like to take the $\mathrm{d}\gamma(x | m)$ term outside, however, this is not easy since $m$ defines $\gamma( \cdot | m)$.

I am applying the following steps, and I am not sure where exactly I am failing since each step looks intuitive but the final expression is counter-intuitive:

\begin{align} & \int_{m \in M} \Big[\int_{x \in \mathbb{R}} c(x) \mathrm{d} \gamma(x | m)\Big] \mathrm{d}\omega(m) \tag{1}\\ = & \int_{m \in M} \Big[\int_{(m',x) \in M \times \mathbb{R}} \chi_{\{ m = m' \}} c(x) \mathrm{d} \gamma(x | m')\Big] \mathrm{d}\omega(m) \tag{2}\\ = & \int_{(m',x) \in M \times \mathbb{R}} c(x) \Big[\underbrace{\int_{m \in M} \chi_{\{ m = m' \}} \mathrm{d}\omega(m)}_{\displaystyle \omega(\{m'\}) ?????} \Big] \mathrm{d} \gamma(x | m'), \tag{3} \end{align} where $\chi$ is the indicator. If this is true, what if $\omega(\{ m' \}) = 0$ everywhere? If this is not true, where am I failing and how can I fix this? Apologies for my lack of measure theory knowledge.

Note: in a similar question, user Masacroso told me this is related to the disintegration theorem. Here I am creating a new question since, (i) I think the question for total expectation generalizes my earlier question, (ii) I am not sure if disintegration is really what I am looking for.

  • 1
    $\begingroup$ you cannot move the term $\mathrm{d}\gamma (x|m)$ outside because it depends on $m$. However you can try to see if the measure define by $\mathrm{d}\nu(x,m):= \mathrm{d}\gamma (x|m)\mathrm{d}\omega (x)$ can be disintegrated in the order of integration that you want, something like $\mathrm{d}\nu (x,m)=\mathrm{d}\alpha (m|x)\mathrm{d}\beta (x)$. However I dont know so much about if this could be possible, this is why I've given just a brief comment about the disintegration theorem in the other question $\endgroup$
    – Masacroso
    Nov 22, 2021 at 7:58


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.