Conditional probability where the conditioning variable is continuous

Consider

• a random variable $Y$ with finite support $\mathcal{Y}$

• a random variable $X$ with cdf $G$ absolutely continuous with probability density function $g$ (i.e., $X$ is a continuous random variable with support $\mathcal{X}$)

• all random variables are defined on the probability space $(\Omega, \mathcal{F}, \mathbb{P})$

For some $y\in \mathcal{Y}$, let $$\mathbb{P}(Y=y| X)\equiv h_y(X)$$ where $h_y: \mathcal{X}\rightarrow [0,1]$

What is the function $$(y,x)\in\mathcal{Y} \times \mathcal{X} \mapsto h_y(x)* g(x)\in \mathbb{R}$$ ? ($*$ denotes scalar multiplication)

Is it the joint probability density function of $(Y,X)$?

• @Michael: I think "$\times$" OP adopted here means scalar multiplication. – hypernova May 9 '18 at 17:01
• It is not a density (for example, we would "sum out" the $y$ variable to get a marginal for $X$, not "integrate out"). But operationally you can certainly say $$P[Y=y] = \int_{x=-\infty}^{\infty} \underbrace{P[Y=y|X=x]}_{h_y(x)}\underbrace{f_X(x)}_{g(x)}dx$$ and also $P[Y=y|X=x]f_X(x) = f_{X|Y}(x|y)P[Y=y]$. – Michael May 9 '18 at 17:07
• Thanks,is it correct to write $\mathbb{P}(Y=y, X\leq \tilde{x})\equiv \int_{x\leq \tilde{x}} h_y(x)g(x)dx$? – TEX May 9 '18 at 17:58
• @CGT: Yes, that is the same as $$P[Y=y, X \leq \tilde{x}] = \int_{x=-\infty}^{\infty} \underbrace{P[Y=y, X\leq \tilde{x}|X=x]}_{P[Y=y|X=x]1\{x\leq \tilde{x}\}}f_X(x)dx$$ I have added a more detailed response below. – Michael May 9 '18 at 19:03

There is no (traditional, i.e., non-impulsive) joint density for $(X,Y)$ because $Y$ is a discrete random variable. Nevertheless, the function $d(x,y) = P[Y=y|X=x]f_X(x)$ can be viewed "operationally" as having the same desirable features of a density, with the understanding that we "sum" over the $y$ variable, not integrate.

• For example, we "sum out $y$" to get the marginal density for $X$: $$\sum_{y \in \mathcal{Y}} P[Y=y|X=x] f_X(x) = f_X(x)$$ This is distinct from "integrating out" the $y$ variable if there were a tranditional density $f_{XY}(x,y)$, i.e., the standard formula $f_X(x) = \int_{y=-\infty}^{\infty} f_{XY}(x,y)dy$.

• Also note that we can "switch the conditioning" as desired: $$P[Y=y|X=x]f_X(x) = f_{X|Y}(x|y)P[Y=y]$$

• Finally, for any measurable set $A \subseteq \mathbb{R}^2$, if we let $1_{\{(x,y) \in A\}}$ be an indicator function that is 1 if $(x,y) \in A$, and zero else, then $$\boxed{P[(X,Y) \in A] = \sum_{y\in \mathcal{Y}} \int_{x=-\infty}^{\infty} P[Y=y|X=x]f_X(x) 1_{\{(x,y) \in A\}} dx}$$ Indeed \begin{align} P[(X,Y)\in A] &= \int_{x=-\infty}^{\infty} P[(X,Y)\in A| X=x] f_X(x)dx\\ &=\int_{x=-\infty}^{\infty} P[(x,Y) \in A|X=x]f_X(x)dx\\ &\int_{x=-\infty}^{\infty} \left(\sum_{y \in \mathcal{Y}} P[Y=y|X=x]1_{\{(x,y)\in A\}}\right) f_X(x)dx \end{align} and we can formally switch sums/integrals by Fubini-Tonelli for this non-negative function.

• Note that my boxed equation may be what is meant by the Mike Earnest equation (2). I do not know what a 1-dimensional Hausdorf measure on $\mathbb{R}^2$ is. – Michael May 9 '18 at 18:58

This relates to a concept I have though about a lot; what follows includes some opinions.

Let $f(x,y)=h_y(x)\cdot g(x)$. You would not call $f$ a joint density for $(X,Y)$. That term is reserved for the situation where $(X,Y)$ is absolutely continuous with respect to Lebesgue measure $\lambda$ on the plane. That is, we cannot say that $$\mathbb P((X,Y)\in A)=\int_A f(x,y)\,d\lambda \tag{1}$$ However, $f$ certainly looks like a density in the following sense. Let $H^1$ denote $1$-dimensional Hausdorf measure on $\mathbb R^2$. Then for any measurable $A$, $$\mathbb P((X,Y)\in A)=\int_A f(x,y)\,dH^1\tag{2}$$ In the situation (1), the support of $(X,Y)$ is a two-dimensional set, wheras in your situation (2), the support is a one dimensional set, a finite union of several lines $\{y\}\times \mathcal X$ for $y\in \mathcal Y$. In other words, if you generalize the notion of "density" to allow for other supports with other dimensions by integrating with respect to Hausdorf measure $H^d$, then you can call your function a (generalized) density. I think that we should adopt this convention, but as far as I know no one does.

Another example: if $X$ has the Cantor distribution, then the function$^*$ $f(x)={\bf 1}(x\in C)$, where $C$ is the Cantor set, can be viewed as a $(\log_3 2)$-dimensional density function for $X$. We then have $$P(X\in A)=\int_A f(x)\,dH^{\log_3 2}$$ This illustrates that $X$ is uniformly distributed over the Cantor set, since the density function is constant.

$^*$ Perhaps you need to normalize constant to make to make this integrate to $1$.