2
$\begingroup$

Consider the following problem, from Tijms's Understanding Probability:

A receiver gets as input a random signal that is represented by a discrete random variable $X$, where $X$ takes on the value +1 with probability $p$ and the value -1 with probability $1- p$. The output $Y$ is a continuous random variable which is equal to the input $X$ plus random noise, where the random noise has an $N (0, \sigma^2 )$ distribution. You can only observe the output. What is the conditional probability of $X = 1$ given the observed value $Y = y$?

This problem is interesting because it seems a mix of discrete and continuous random variables. My attempt.

For the probability that $Y\le y$, we have two possibilities: either $X=1$ or $X=-1$, and $P(Y\le y) = P(Y\le y|X=1)P(X=1) + P(Y\le y|X=-1)P(X=-1)$, so $$ P(Y\le y)=p\int_{-\infty}^y \frac{1}{\sqrt{2\pi}}e^{-\frac12 (s-1)^2}ds + (1-p)\int_{-\infty}^y \frac{1}{\sqrt{2\pi}}e^{-\frac12 (s+1)^2}ds. $$ Here I used the fact that: $$ P(Y\le y|X=1) = \int_{-\infty}^y \frac{1}{\sqrt{2\pi}}e^{-\frac12(s-1)^2}ds. $$ Then, we might use: $$ P(X=1|Y=y) = P(Y=y|X=1)\frac{P(Y= y)}{P(X=1)}, $$ however if I replace the terms $P(Y\le y)$ and $P(Y\le y|X=1)$ I derived earlier, first I get a complicated simple expression, and then I am puzzled by having something like $P(Y=y)$ in the context of a continuous random variable.

$\endgroup$
2
$\begingroup$

I know this sounds like nonsense, but for conditioning on a continuous variable $Y$ having a particular value $Y=y$, you can simply substitute the pdf where you would normally have $P(Y=y)$ (whose value is technically $0$).

Indeed, your situation is exactly Bayes theorem with one discrete variable and one continuous variable.

I don't know enough of the underlying theory (presumably measure theory?) to explain why this works. Sorry!

$\endgroup$
1
  • $\begingroup$ in probability without measure theory, it's actually a definition? $\endgroup$ Mar 24 at 7:01
1
$\begingroup$

Consider a mixed joint probability distribution

$$P(X=x, Y=y) \approx P(X=x)P(Y \in y+\delta y|X=x) =\\ P(X=x|Y \in y + \delta y)P(Y \in y+ \delta y) $$

For $\delta y \rightarrow 0$, we can write:

$$P(X=x)f_Y(y|X=x) = P(X=x|Y = y)f_Y(y) $$

Then, $$P(X=1|Y = y) = \frac{pf_Y(y|X=1)}{f_Y(y)}, $$

where $f_Y(y|X=1) = p N(1,\sigma^2)$ and $f_Y(y) = p N(1,\sigma^2) + (1-p)N(-1,\sigma^2)$, and $N(\mu,\sigma^2)$ is a density of a normal r.v. with mean $\mu$ and variance $\sigma^2$.

$\endgroup$
0
$\begingroup$

Following the example by dnqxt, I suspect that we can interpret: $$ P(Y=y) = \lim_{\delta \to 0}P(y-\delta \le Y\le y) =\lim_{\delta\to 0} \left(p\frac{\delta}{\sqrt{2\pi}}e^{-\frac12 (y-1)^2} + (1-p)\frac{\delta}{\sqrt{2\pi}}e^{-\frac12(y+1)^2}\right), $$ and similarly, $$ P(X=1,Y=Y)=\lim_{\delta\to0}P(X=1, y-\delta\le Y\le y)=\lim_{\delta\to0}p\frac{\delta}{\sqrt{2\pi}}e^{-\frac12 (y-1)^2}. $$ As expected, $P(Y=y)$ and $P(X=1,Y=y)$ are zero when we take the limit, but computing the conditional probability before taking the limit gives: $$ P(X=1|Y=y) = \frac{pe^{-\frac12 (y-1)^2}}{pe^{-\frac12 (y-1)^2}+(1-p)e^{-\frac12 (y+1)^2}}. $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.