• a random variable $Y$ with finite support $\mathcal{Y}$

  • a random variable $X$ with cdf $G$ absolutely continuous with probability density function $g$ (i.e., $X$ is a continuous random variable with support $\mathcal{X}$)

  • all random variables are defined on the probability space $(\Omega, \mathcal{F}, \mathbb{P})$

For some $y\in \mathcal{Y}$, let $$ \mathbb{P}(Y=y| X)\equiv h_y(X) $$ where $h_y: \mathcal{X}\rightarrow [0,1]$

What is the function $$ (y,x)\in\mathcal{Y} \times \mathcal{X} \mapsto h_y(x)* g(x)\in \mathbb{R} $$ ? ($*$ denotes scalar multiplication)

Is it the joint probability density function of $(Y,X)$?

  • $\begingroup$ @Michael: I think "$\times$" OP adopted here means scalar multiplication. $\endgroup$ – hypernova May 9 '18 at 17:01
  • $\begingroup$ It is not a density (for example, we would "sum out" the $y$ variable to get a marginal for $X$, not "integrate out"). But operationally you can certainly say $$ P[Y=y] = \int_{x=-\infty}^{\infty} \underbrace{P[Y=y|X=x]}_{h_y(x)}\underbrace{f_X(x)}_{g(x)}dx$$ and also $P[Y=y|X=x]f_X(x) = f_{X|Y}(x|y)P[Y=y]$. $\endgroup$ – Michael May 9 '18 at 17:07
  • $\begingroup$ Thanks,is it correct to write $\mathbb{P}(Y=y, X\leq \tilde{x})\equiv \int_{x\leq \tilde{x}} h_y(x)g(x)dx$? $\endgroup$ – TEX May 9 '18 at 17:58
  • $\begingroup$ @CGT: Yes, that is the same as $$P[Y=y, X \leq \tilde{x}] = \int_{x=-\infty}^{\infty} \underbrace{P[Y=y, X\leq \tilde{x}|X=x]}_{P[Y=y|X=x]1\{x\leq \tilde{x}\}}f_X(x)dx $$ I have added a more detailed response below. $\endgroup$ – Michael May 9 '18 at 19:03

There is no (traditional, i.e., non-impulsive) joint density for $(X,Y)$ because $Y$ is a discrete random variable. Nevertheless, the function $d(x,y) = P[Y=y|X=x]f_X(x)$ can be viewed "operationally" as having the same desirable features of a density, with the understanding that we "sum" over the $y$ variable, not integrate.

  • For example, we "sum out $y$" to get the marginal density for $X$: $$\sum_{y \in \mathcal{Y}} P[Y=y|X=x] f_X(x) = f_X(x)$$ This is distinct from "integrating out" the $y$ variable if there were a tranditional density $f_{XY}(x,y)$, i.e., the standard formula $f_X(x) = \int_{y=-\infty}^{\infty} f_{XY}(x,y)dy$.

  • Also note that we can "switch the conditioning" as desired: $$ P[Y=y|X=x]f_X(x) = f_{X|Y}(x|y)P[Y=y]$$

  • Finally, for any measurable set $A \subseteq \mathbb{R}^2$, if we let $1_{\{(x,y) \in A\}}$ be an indicator function that is 1 if $(x,y) \in A$, and zero else, then $$\boxed{P[(X,Y) \in A] = \sum_{y\in \mathcal{Y}} \int_{x=-\infty}^{\infty} P[Y=y|X=x]f_X(x) 1_{\{(x,y) \in A\}} dx}$$ Indeed \begin{align} P[(X,Y)\in A] &= \int_{x=-\infty}^{\infty} P[(X,Y)\in A| X=x] f_X(x)dx\\ &=\int_{x=-\infty}^{\infty} P[(x,Y) \in A|X=x]f_X(x)dx\\ &\int_{x=-\infty}^{\infty} \left(\sum_{y \in \mathcal{Y}} P[Y=y|X=x]1_{\{(x,y)\in A\}}\right) f_X(x)dx \end{align} and we can formally switch sums/integrals by Fubini-Tonelli for this non-negative function.

  • $\begingroup$ Note that my boxed equation may be what is meant by the Mike Earnest equation (2). I do not know what a 1-dimensional Hausdorf measure on $\mathbb{R}^2$ is. $\endgroup$ – Michael May 9 '18 at 18:58

This relates to a concept I have though about a lot; what follows includes some opinions.

Let $f(x,y)=h_y(x)\cdot g(x)$. You would not call $f$ a joint density for $(X,Y)$. That term is reserved for the situation where $(X,Y)$ is absolutely continuous with respect to Lebesgue measure $\lambda$ on the plane. That is, we cannot say that $$ \mathbb P((X,Y)\in A)=\int_A f(x,y)\,d\lambda \tag{1} $$ However, $f$ certainly looks like a density in the following sense. Let $H^1$ denote $1$-dimensional Hausdorf measure on $\mathbb R^2$. Then for any measurable $A$, $$ \mathbb P((X,Y)\in A)=\int_A f(x,y)\,dH^1\tag{2} $$ In the situation (1), the support of $(X,Y)$ is a two-dimensional set, wheras in your situation (2), the support is a one dimensional set, a finite union of several lines $\{y\}\times \mathcal X$ for $y\in \mathcal Y$. In other words, if you generalize the notion of "density" to allow for other supports with other dimensions by integrating with respect to Hausdorf measure $H^d$, then you can call your function a (generalized) density. I think that we should adopt this convention, but as far as I know no one does.

Another example: if $X$ has the Cantor distribution, then the function$^*$ $f(x)={\bf 1}(x\in C)$, where $C$ is the Cantor set, can be viewed as a $(\log_3 2)$-dimensional density function for $X$. We then have $$ P(X\in A)=\int_A f(x)\,dH^{\log_3 2} $$ This illustrates that $X$ is uniformly distributed over the Cantor set, since the density function is constant.

$^*$ Perhaps you need to normalize constant to make to make this integrate to $1$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.