# Conditional probability combining discrete and continuous random variables

Consider the following problem, from Tijms's Understanding Probability:

A receiver gets as input a random signal that is represented by a discrete random variable $$X$$, where $$X$$ takes on the value +1 with probability $$p$$ and the value -1 with probability $$1- p$$. The output $$Y$$ is a continuous random variable which is equal to the input $$X$$ plus random noise, where the random noise has an $$N (0, \sigma^2 )$$ distribution. You can only observe the output. What is the conditional probability of $$X = 1$$ given the observed value $$Y = y$$?

This problem is interesting because it seems a mix of discrete and continuous random variables. My attempt.

For the probability that $$Y\le y$$, we have two possibilities: either $$X=1$$ or $$X=-1$$, and $$P(Y\le y) = P(Y\le y|X=1)P(X=1) + P(Y\le y|X=-1)P(X=-1)$$, so $$P(Y\le y)=p\int_{-\infty}^y \frac{1}{\sqrt{2\pi}}e^{-\frac12 (s-1)^2}ds + (1-p)\int_{-\infty}^y \frac{1}{\sqrt{2\pi}}e^{-\frac12 (s+1)^2}ds.$$ Here I used the fact that: $$P(Y\le y|X=1) = \int_{-\infty}^y \frac{1}{\sqrt{2\pi}}e^{-\frac12(s-1)^2}ds.$$ Then, we might use: $$P(X=1|Y=y) = P(Y=y|X=1)\frac{P(Y= y)}{P(X=1)},$$ however if I replace the terms $$P(Y\le y)$$ and $$P(Y\le y|X=1)$$ I derived earlier, first I get a complicated simple expression, and then I am puzzled by having something like $$P(Y=y)$$ in the context of a continuous random variable.

I know this sounds like nonsense, but for conditioning on a continuous variable $$Y$$ having a particular value $$Y=y$$, you can simply substitute the pdf where you would normally have $$P(Y=y)$$ (whose value is technically $$0$$).

Indeed, your situation is exactly Bayes theorem with one discrete variable and one continuous variable.

I don't know enough of the underlying theory (presumably measure theory?) to explain why this works. Sorry!

• in probability without measure theory, it's actually a definition? Mar 24 at 7:01

Consider a mixed joint probability distribution

$$P(X=x, Y=y) \approx P(X=x)P(Y \in y+\delta y|X=x) =\\ P(X=x|Y \in y + \delta y)P(Y \in y+ \delta y)$$

For $$\delta y \rightarrow 0$$, we can write:

$$P(X=x)f_Y(y|X=x) = P(X=x|Y = y)f_Y(y)$$

Then, $$P(X=1|Y = y) = \frac{pf_Y(y|X=1)}{f_Y(y)},$$

where $$f_Y(y|X=1) = p N(1,\sigma^2)$$ and $$f_Y(y) = p N(1,\sigma^2) + (1-p)N(-1,\sigma^2)$$, and $$N(\mu,\sigma^2)$$ is a density of a normal r.v. with mean $$\mu$$ and variance $$\sigma^2$$.

Following the example by dnqxt, I suspect that we can interpret: $$P(Y=y) = \lim_{\delta \to 0}P(y-\delta \le Y\le y) =\lim_{\delta\to 0} \left(p\frac{\delta}{\sqrt{2\pi}}e^{-\frac12 (y-1)^2} + (1-p)\frac{\delta}{\sqrt{2\pi}}e^{-\frac12(y+1)^2}\right),$$ and similarly, $$P(X=1,Y=Y)=\lim_{\delta\to0}P(X=1, y-\delta\le Y\le y)=\lim_{\delta\to0}p\frac{\delta}{\sqrt{2\pi}}e^{-\frac12 (y-1)^2}.$$ As expected, $$P(Y=y)$$ and $$P(X=1,Y=y)$$ are zero when we take the limit, but computing the conditional probability before taking the limit gives: $$P(X=1|Y=y) = \frac{pe^{-\frac12 (y-1)^2}}{pe^{-\frac12 (y-1)^2}+(1-p)e^{-\frac12 (y+1)^2}}.$$