I have been working on the reconciliation of the treatment of discrete and continuous random variable in a measure theoretic sense. But I found myself blocked on fundamental results.

We know that If $X$ and $Y$ are respectively discrete and continuous we have the marginals : \begin{align} P(A) = \sum_i P(A,X_i)\\ p(A) = \int_{\Omega_Y} p(A,t)dt\\ \end{align}

I am trying to find the equation that generalize these results and by derivation find those actual results.

Since the discrete case involve measures and the continuous case involve densities I don't know where to start.

I could use the density of the discrete variable using Dirac but then again I am stuck because to derive the continuous case we need the Radon-Nikodym theorem which is not available because $X$ is not absolutely continuous to the lesbegue measure.

1) Do I need to consider a special measure for the discrete case ?

2) Do I miss any layer of abstraction here that would allow me to derive these results ?

Thank you for any pointer !


Part of my question is also how to define the marginal of a joint distribution if we don't know if the second random variable is discrete or continuous. If it is discrete we can define it in term of sum and if it is continuous we can define it in term of an integral of the joint density.

But in the general case how do we define it ( not knowing if discrete or continuous ).

Above notation :

$P(A) = P(a\in A, x \in \Omega_X)$ so $P$ is a probability distribution

$P(a \in A) = \int_A p(x)dx$ so $p$ is a density

  • $\begingroup$ Indeed, this is possible; could you just clarify your notation? Let's say you are given a joint distribution $\Bbb P$, what is $P(A)$, $P(A,X_i)$, $p(A)$ and $p(A,t)$ in terms of $\Bbb P$? $\endgroup$ – Ilya Dec 9 '14 at 16:06
  • $\begingroup$ Thank you for your answer. I put some additional comments on my original question. $\endgroup$ – user149705 Dec 9 '14 at 17:33
  • $\begingroup$ I was more concerned with the functions of two arguments that appear behind the sum and integral, please be precise with them $\endgroup$ – Ilya Dec 9 '14 at 18:15
  • $\begingroup$ that would be the joint probability distribution and the joint density respectively. $\endgroup$ – user149705 Dec 9 '14 at 19:37

Your notation is still a bit imprecise, but the message is pretty much clear. Let $\Bbb P$ be a product distribution, that is some probability measure on the product space $\Omega\times \bar\Omega$. The marginals of $\Bbb P$ on each factor spaces are given (be definition) as $$ \mathsf P(A):=\Bbb P(A\times \bar\Omega)\qquad \bar{\mathsf P}(B) := \Bbb P(\Omega\times B) $$ for all measurable $A$ and $B$. Now, is the factor spaces are nice measurable spaces (e.g. $\Bbb R^n$ with Borel $\sigma$-algebra) then there always exist conditional distributions such that $$ \Bbb P(A\times B) = \int_\Omega\kappa(B|\omega)\mathsf P(\mathrm d\omega) = \int_{\bar\Omega}\bar\kappa(A|\bar\omega)\bar{\mathsf P}(\mathrm d\bar\omega) \tag{1} $$ for all measurable $A$ and $B$, and $(1)$ determines $\Bbb P$ uniquely since to construct the product measure we have to define it (in a consistent way) just on measurable rectangles.

Consider now the two special cases that you have.

  1. The fact that the first variable is discrete means that $\mathsf P = \sum_i\mathsf P(\omega_i)\delta_{\omega_i}$ for some countable collection of points $\omega_i\in \Omega$. In that case the conditional distribution does always exist, so the formula $(1)$ boils down to $$ \bar{\mathsf P}(B) = \int_\Omega\kappa(B|\omega)\mathsf P(\mathrm d\omega) = \sum_{i}\mathsf P(\omega_i)\kappa(B|\omega_i) = \sum_{i}\mathsf P(\omega_i)\kappa(B|\omega_i) = \sum_i \Bbb P(\{\omega_i\}\times B). $$

  2. For the continuous random variable, we have $\bar\Omega = \Bbb R$ and $\bar{\mathsf P} \ll \lambda$ where $\lambda$ is the Lebesgue measure, so let $\bar p$ be its density. From $(1)$ we obtain: $$ \mathsf P(A) = \int_{\Bbb R}\bar\kappa(A|\bar\omega)\bar{\mathsf P}(\mathrm d\bar\omega) = \int_{\Bbb R}\bar\kappa(A|\bar\omega)\bar p(\bar\omega)\lambda(\mathrm d\bar\omega) $$ which recovers your second formula. So, that's why I was wondering what does "$p(A,t)$" mean in your case: the joint density (w.r.t Lebesgue measure) may not exist: for sure it does not exist if the first variable has discrete distribution. Not to say that density does not have set-valued arguments. So in fact we are talking about conditional distribution multiplied by the density of the second variable. If both variables are continuous and the joint density $q$ does exist, then your "$p(A,t)$" is $\int_A q(s,t)\lambda(\mathrm ds)$.

  • $\begingroup$ Thank you for your answer. I have couple questions : 1) I am not familiar with the (1) formula. I am aware of the definition of product measure as an integral of a two parameters measurable function but not in term of conditional distribution. The result I know is : \begin{align} \lambda(AxB) = \int_A f_x(B)\mu(dx) \end{align} Could you provide me some intuition or reference on where to derive this result with a conditional probability distribution ? I am still trying to work out the rest .... $\endgroup$ – user149705 Dec 10 '14 at 18:01
  • $\begingroup$ @user149705: you just wrote exactly the formula $(1)$; what is $f_x(B)$ you think? $\endgroup$ – Ilya Dec 10 '14 at 18:35
  • $\begingroup$ as i understood it, it would be a function of two parameters. i dont understand where the constraint of it being conditional comes from. Or is it just a notation issue ? Anyways i will continue to work out the end of your answer :) $\endgroup$ – user149705 Dec 10 '14 at 19:44
  • $\begingroup$ I just finished figuring out your explanation. Let me say that you helped a lot, if you read this thank you ! I was able to derive every case discrete/discrete, discrete/continuous, continuous/discrete and continuous/continuous. I just need to find out why if the spaces are measurable we get for sure the equation (1). Thank you again ! $\endgroup$ – user149705 Dec 18 '14 at 18:15
  • $\begingroup$ @user149705: glad to be of help. Your last point is not really trivial. Given a marginal $\mathsf P$ and a conditional distribution function $\kappa$ you can construct the joint $\Bbb P$: this is easy to prove, e.g. see the "Probability" by Ash. However, getting $\kappa$ from $\Bbb P$ is much harder: this is called the problem of existence of regular conditional probability (RCP) and it only holds for nice spaces (like Borel subsets of $\Bbb R^n$). $\endgroup$ – Ilya Dec 19 '14 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.