# Conditional Probability of $P(X|X^2)$ for uniform distribution.

Assume $$X \sim U[-1,1]$$ is a uniform distribution. We can tell intuitively that $$P(X=k|X^2=x^2) = \begin{cases} 1/2 \;\;\;\;\;\;\;\text{if} \;\; k=-x \\ 1/2 \;\;\;\;\;\;\;\text{if} \;\; k=x \end{cases}$$

But I want to confirm this using regular conditional probability. However, when I try the definition as $$P(X\in A|Y=y):=\lim\limits_{\{Y=y\}\in U}\frac{P(A\cap U)}{P(U)}$$, the conditional probability is $$0$$ since for example if trying to compute $$P(X=2|X^2=4):=\lim\limits_{\epsilon \to 0}\frac{P(X=2 \;\cap\; X^2 \in [4-\epsilon,4+\epsilon])}{P(X^2 \in [4-\epsilon,4+\epsilon])}$$, which is just $$0$$ because $$X$$ is continuous distribution. Maybe I got something wrong. Please help me compute this conditional probability formally.

• What are U and A? Being too terse is confusing. Nov 5, 2020 at 21:28
• Sorry for that, I just made the question more clear by adding some computations. Nov 5, 2020 at 22:10
• You're actually conditioning on the outcome that $X$ belongs to the set $\{-x,x\}$ which makes me wonder... If $X\sim f_X$ is a continuous random variable supported on all of $\mathbb{R}$ and $S\subseteq \mathbb{R}$ is a finite set, then for $s\in S$ fixed do we have $$P(X=s|X\in S)=\frac{f_X(s)}{\sum_{x\in S}f_X(x)}$$ Nov 5, 2020 at 23:19

I think what you're trying to do is to find conditional law for $$X$$ and this one question is closely related to the problem of probability kernel.
Theorem
Let $$(E_1, \mathcal{E}_1 )$$ and $$(E_2, \mathcal{E}_2 )$$ be two measurable function. Suppose further that $$(E_2, \mathcal{E}_2 )$$ is regular ( Remark: $$\mathbb{R}$$ is regular).
Let $$m$$ be a finite positive measure on $$E_1 \otimes E_2$$, with $$\mu$$ as its first marginal law.
Then there is a unique (in almost sure sense) probability kernel from $$(E_1, \mathcal{E}_1 )$$ to $$(E_2, \mathcal{E}_2 )$$ such that $$m= \mu \otimes p$$, that is:
$$\int_{E_1 \times E_2} gdm= \int_{E_1} \mu(dx) \int_{E_2} p(x, dy)g(x,y)$$
for all measureable function $$g:E_1\times E_2 \longrightarrow \mathbb{R_+}$$
In particular, we have:
Corollary:
$$\mathbb{E}(f(Y) | X) = \int_{\mathbb{R}} f(y) p(X,dy)$$ for all positive measureable function $$f$$, a.s
where $$p$$ is the respective kernel for $$m= \mathcal{L}(X,Y)$$

Remark 1 In fact, there are a lot of technical questions in these statements.(measurability, regularity, almost sureness, spaces of measures, etc.) So if you may, please don't be concerned too much of those because it'll be a heavy burden on you.
Remark 2 Roughly speaking, $$p$$ represents conditional law.
Remark 3 $$p(X=2|X^2=2)$$ doesn't mean anything because $$P(X^2=2)=0$$.We can replace it by anything we want, so "calculating it" also doesn't mean anything. But $$\mathcal{L}(X| X^2)$$ is a different thing. That's why we introduce the notion of probability kernel.

Back to your question, what you're trying to find is the probability kernel for $$(X^2,X)$$
. You can find it by your intuition or anything and check it with the unicity in the above theorem ( yeah, this is the right way of *calculating it* in my opinion) then use it in form of the above corollary when you want.

**Appendix **
$$\mathcal{L}(X)$$: law of $$X$$.

Disclaim any errors in these statements are mine any maybe due to my reluctance to check the question of almost sureness.

• That was very helpful. So I just wanna check one more thing to see if my understanding is correct: So by conditional pdf (or conditional pmf), you are suggesting the probability kernel is the right thing to look like, since the integral of that will give you the conditional expectation which is well-defined. is that correct ? Nov 9, 2020 at 5:50
• also, what's the relationship between probability kernel with regular conditional probability? Nov 9, 2020 at 5:52
• your saying is almost like my opinion, but I would like to emphasize the fact that: when dealing with the conditional law of a continuously distributed variable, we are obliged to look at a whole family of conditional law. The main reason is not because of well-definedness but the almost sureness. It's similar to why when we talking about measurable functions, we are not much concerned about what happens at a particular point. Nov 9, 2020 at 6:08
• About regular conditional probability, I have to admit that never ever in my life I have used that term. So not much knowledge from me, however, it seems to me regular condtional probability and probability kernel are two notions that are created for the same goal to describe the conditional law. Nov 9, 2020 at 6:10
• While one ( regular conditional probability) is much concerned about the initial probability space ( $(\Omega, P)$ ), the other only cares about the conditional law between two variables. To me, I like the latter. Too many burdens for a notion is not my thing. Nov 9, 2020 at 6:13

I'm not convinced something like $$P(X=1|X^2=1)$$ is well$$-$$defined if $$X\sim \mathcal{U}(-2,2)$$, and it's not because $$P(X^2=1)=0$$; it is possible (in fact, common) to condition on events with vanishing probabilities in higher$$-$$dimensional space. For example, if $$(X,Y)\sim f_{XY}$$ is a continuous random vector that's supported on $$\mathbb{R}^2$$, then $$P(X<0|Y=X^2)=\frac{\int_{-\infty}^{0}f(t,t^2)\sqrt{4t^2+1}dt}{\int_{-\infty}^{\infty}f(t,t^2)\sqrt{4t^2+1}dt}$$ even though $$P(Y=X^2)=0$$.

Here is why I don't believe something like $$P(X=1|X^2=1)$$ is well defined when $$X \sim \mathcal{U}(-2,2)$$. Notice how conditioning on $$\{X^2=1\}$$ is equivalent to conditioning on the event that $$X$$ belongs to the finite set $$\{-1,1\}$$. That being said, define intervals $$I_1,I_2\subseteq (-2,2)$$ by $$I_1=(-1-\epsilon_1,-1+\epsilon_1)$$ and $$I_2=(1-\epsilon_2,1+\epsilon_2)$$ where $$\epsilon_1,\epsilon_2$$ are very small positive real numbers. Notice $$P(X\in I_2|X\in I_1 \cup I_2)=\frac{\epsilon_2}{\epsilon_1 +\epsilon_2}$$ Now if $$P(X=1|X^2=1)$$ were well$$-$$defined, we would certainly have $$P(X=1|X^2=1)=\lim_{(\epsilon_1,\epsilon_2)\rightarrow (0^+,0^+)}\bigg(\frac{\epsilon_2}{\epsilon_1+\epsilon_2}\bigg)$$ But this limit doesn't exist.

My original intuition made me think that if $$X\sim f_X$$ is a continuous random variable supported on $$\mathbb{R}$$ and if $$S\subseteq \mathbb{R}$$ is a finite set, then for any $$s\in S$$ we have $$P(X=s|X\in S)=\frac{f_X(s)}{\sum_{x\in S}f_X{(x)}}$$ but I no longer believe this is the case based off my previous comments. Any thoughts?

• It is possible that if two random variables are jointly continuous but one conditional on the other is discrete then we cannot have the usual decomposition $p(x,y) = p(x|y) p(y)$ because two are probability density functions and one (the conditional) is a probability mass function. I may have another similar example from the two envelope problem. I'll see if I can recover it from my memory. Nov 6, 2020 at 2:57
• I found this helpful. He didn't talk about conditional distribution though. probabilitycourse.com/chapter4/4_3_2_delta_function.php Nov 6, 2020 at 15:25
• Have you found an example where $$p_{XY}(x,y) \neq p_{X|Y}(x|y)p_{Y}(y)?$$I'd be very interested to see that example. Nov 7, 2020 at 5:13
• After reading about the Dirac delta function I no longer think it's relevant because you just cannot mix a pdf with a pmf. But I don't know if the "generalized pdf" will have the same nice decomposition. Please take a look at my updated answer. Nov 7, 2020 at 16:56
• Thank you for both of your answers! I found that @Paresseux Nguyen 's answer was very helpful. I think what I'm looking for is the "probability kernel", and it should be the "conditional pdf" in the sense that if you integral this kernel to compute expectation, it gives you the conditional expectation. : ) Nov 9, 2020 at 5:48

Since $$U(-1,1)$$ is a continuous distribution, $$\Pr(X=k|X^2=x^2)$$ is not well defined. Instead you first find the CDF given $$u \leqslant X^2 \leqslant u+\Delta u$$, for some small $$\Delta u$$ (you can assume it's positive), then take the derivative w.r.t. $$k$$, and let $$\Delta u \to 0$$ to get your conditional PDF.

So in this case your unconditional probability is $$\Pr(u \leqslant X^2 \leqslant u+\Delta u)=\sqrt{u+\Delta u} - u. \tag 1$$

Then you need to calculate $$\Pr(X \leqslant k \text{ and } u \leqslant X^2 \leqslant u+\Delta u).\tag 2$$

This is a piece-wise linear function of $$k$$. I will leave the details to you.

Finally you calculate the ratio of (2) to (1), take the derivative, let $$\Delta u \to 0$$, you will get your conditional probability mass function.

============ Update:

I found this interesting: https://www.probabilitycourse.com/chapter4/4_3_2_delta_function.php

It basically says that the "symbolic derivative" of the Heavyside step function $$u(x)$$ is the Dirac delta function $$\delta(x)$$, where

$$u(x) = \begin{cases} 1, & x\geqslant 0 \\ 0, & x < 0 \end{cases}$$

and $$\delta(x) = \begin{cases} \infty, & x=0 \\ 0, & \text{elsewhere} \end{cases}$$

And any discrete distribution has a generalized PDF in terms of the delta function: $$f_X(x) = \sum \Pr(X=x_k) \delta (x-x_k).$$

Now back to OP's question. We look at the joint distribution of $$(X,Y)$$ on $$[-1,1] \times [0,1]$$, where $$X \sim U[-1,1], Y=X^2, a.s.$$

On one hand, $$f_{X|Y}(x | y) = \frac 12 \left( \delta(x+\sqrt y) + \delta(x-\sqrt y) \right)$$

$$f_Y(y) = \frac{1}{2\sqrt y}$$

On the other hand, $$F_{X,Y} (x,y) = \Pr(X\leqslant x, X^2\leqslant y)= \Pr\left(-\sqrt y \leqslant X\leqslant \min(x,\sqrt y)\right) \\ = \begin{cases} \frac{x+\sqrt{y}}{2}, & x < \sqrt y\\ \sqrt y, & x>\sqrt y \end{cases}\\ = \frac{x+\sqrt y}{2} - \frac{x-\sqrt y}{2} u(x-\sqrt y)$$

We want to show that

$$f_{X,Y} (x,y) = f_{X|Y} (x|y) f_Y(y)$$

But the link I provided did not discuss the multivariate distribution case. Getting $$f_{X,Y} (x,y)$$ involves taking the derivative of $$F_{X,Y}(x,y)$$ w.r.t. to $$x$$ and $$y$$, which in term requires the derivative of the delta function. I only found this: https://en.wikipedia.org/wiki/Unit_doublet but there is little information.

• Can you explain a little bit why $P(X=k|X^2=x^2)$ is not well defined? It is possible, for example, to condition on events with vanishing probabilities in higher dimenional space. For example, if $(X,Y)\sim f_{XY}$ is a continuous random vector supported on all of $\mathbb{R}^2$ then an expression like $P(X<0|Y=X^2)$ is well defined. Nov 5, 2020 at 23:13
• In fact, $$P(X<0|Y=X^2)=\frac{\int_{-\infty}^{0}f_{XY}(t,t^2)\sqrt{1+4t^2}dt }{\int_{-\infty}^{\infty}f_{XY}(t,t^2)\sqrt{1+4t^2}dt}$$ Nov 5, 2020 at 23:25
• Thanks for your comments. What I meant was it's not well defined in the usual sense of $\Pr (A|B) = \frac{\Pr (A\cap B)}{\Pr(B)}$. The calculation I provided was, in a sense, the same as yours. Nov 6, 2020 at 0:38
• thank you so much for the comments, and link to the file. I'm not sure if you heard of regular conditional probability before but that is the most similar thing i was looking at in the beginning... Nov 9, 2020 at 5:51
• No I have not heard, thank you for letting me know! I checked Wikipedia. From the example there (en.wikipedia.org/wiki/Regular_conditional_probability#Example) it's doing exactly what I did in the first part of my answer. Nov 9, 2020 at 14:23