# conditional probability on zero probability events and conditional Radon-Nikodym derivatives

Consider a stochastic process $\{x_t\}_{t\in T}$ adapted to some filtered probability space $(\Omega,\mathcal{F},\{\mathcal{F}\}_{t\in T},\mathbb{P})$ taking values in the state space $(\mathbb{R},\mathcal{B})$

I wish to consider the probability Pr$(x_t\in\mathcal{A}|x_s)$ where $(s<t)$

This should be a sensible question, as I should be able to assign probability to a question of the form "What is the chance I obtain an outcome $x_t \in [0.5,0.6]$ given that last time I got $x_s=0.4$". e.g. a transition probability.

Naturally we have Pr$(A| B)=$Pr$(A\cap B)/$Pr$(B)$ but then we have Pr$(B)=$Pr$(x_s)=0$ for any particular value.

Now I think that this is where we have the notion of regular conditional probability entering which, as I understand it means we write:

$$\text{Pr}(x_t\in \mathcal{A}|x_s=B)=\lim_{\mathcal{B}\to B}\frac{\text{Pr}(x_t\in \mathcal{A}\cap x_s \in \mathcal{B})}{\text{Pr}(x_s\in \mathcal{B})}$$

Vagaries of the meaning of $\lim_{\mathcal{B}\to B}$ aside (which would have to be implementation specific e.g. here $\lim_{r\to 0} \text{Pr}(x_s\in (B-r,B+r)$), is the above correct?

Should I understand this as a Radon Nikodym derivative? It seems related, but not identical.

How do I relate this to, and formulate it in such a way to be consistent to, how conditional probabilities are usually defined? (as I understand it) viz

$$\text{Pr}(x_t\in\mathcal{A}|x_s\in\mathcal{B})=\mathbb{E}_{\mathbb{P}}[1_{\mathcal{A}}(x_t)|\sigma(\mathcal{B})\subseteq\mathcal{F}_s]$$

Surely $$\mathbb{E}_{\mathbb{P}}[1_{\mathcal{A}}(x_t)|\sigma({B})]$$ just wouldn't work? i.e. $\sigma(B)\nsubseteq\mathcal{F}$?

Is it legitimate to construct Radon-Nikodym derivatives out of measures formed from regular conditional probabilities? i.e. is this (heuristically) ok?

$$\frac{d\mathbb{P}(x_t|x_s=B)}{d\mathbb{Q}(x_t|x_s=B)}=\lim_{\mathcal{A}\to\emptyset}\lim_{\mathcal{B}\to B}\frac{\mathbb{P}(x_t\in\mathcal{A}|x_s\in\mathcal{B})}{\mathbb{Q}(x_t\in\mathcal{A}|x_s\in\mathcal{B})}$$

Thanks.

EDIT:

Based on discussion in the comments, the issues appears to boil down to why one can write

$$\text{Pr}(x_t\in \mathcal{A}|B)=\mathbb{E}_{\mathbb{P}}[1_{\mathcal{A}}(x_t)|\sigma({B})]$$

when $B$ is a zero probability event. What I don't understand is what the sigma algebra generated by a zero probability event looks like and why you can condition on it.

Surely the sigma algebra generated by a zero probability event is itself formed from (complements and unions of) zero probability events in $\mathbb{P}$, thus not in $\mathcal{F}$

e.g. $$\sigma(x_s=B)=\{\omega,\omega^c,\emptyset,\Omega\}\nsubseteq\mathcal{F}$$ with $\mathbb{P}(\omega)=0$ such that $\mathbb{E}_{\mathbb{P}}[f(x_t)|\sigma({B})]=\mathbb{E}_{\mathbb{P}}[f(x_t)]$ or $0$?

so why can we condition on these?

What am I getting wrong here?

• It seems the same question could be simplified to just considering a random vector $(X,Y)$, and defining $P[X \in A | Y=y]$, no? The stochastic process and filtration stuff seems like a distraction from the main concepts here. – Michael Jul 23 '16 at 14:00
• A non-measure theory version of probability might "define" this by flipping the conditioning, so $P[X\in A| Y=y] = \frac{f_{Y|X \in A}(y)P[X \in A]}{f_Y(y)}$, assuming $f_Y(y)\neq 0$ and $f_{Y|X \in A}(y)$ makes sense. – Michael Jul 23 '16 at 14:03
• Ok, so I'm simplifying what I need this for (which ultimately is going to be random functions 'given' other functions), hence the desire for a proper measure theoretic understanding, but was hoping to streamline the discussion with the above. Granted you can do the above with probability densities. – user3353819 Jul 23 '16 at 14:16
• If you are comfortable with expectations given sigma algebras, why not just use $P[X \in A|\sigma(Y)] = E[1_A(X)|\sigma(Y)]$? – Michael Jul 23 '16 at 14:17
• Perhaps this is where my intuition breaks. Is conditioning on $\sigma(Y=y)$ legitimate and if so, as you suggest, why? I understand conditioning on $\sigma(\mathcal{Y})$ where $Y\in\mathcal{Y}$. – user3353819 Jul 23 '16 at 14:23

Just a summary of my comments that may clarify: You "should never" consider the sigma algebra generated by an event (since it has nothing to do with expressions like $E[X|Y=y]$). You "should" consider the sigma algebra generated by a random variable, and this has lots to do with expressions like $E[X|Y=y]$. Specifically...

Let $S$ be a sample space. Let $(X,Y)$ be a random vector that maps $S\rightarrow \mathbb{R}^2$. So $X(\omega)$ and $Y(\omega)$ are real numbers for all $\omega \in S$.

### Useful:

$$\sigma(Y) = \mbox{Sigma algebra generated by random variable Y}$$

Then $\sigma(Y)$ has many events, including all events of the form $\{Y\leq y\}$ and $\{Y \in [y, y+\delta]\}$. Now, $E[X|Y]$ (sometimes written $E[X|\sigma(Y)]$) is a random variable with certain properties. There are different "versions," but they all differ only on a set of measure 0. Let $Z=E[X|Y]$ be a particular version. It is "$Y$-measurable" and:

(i) $Z(\omega) = f(Y(\omega))$ for some function $f$ and for all $\omega \in S$.

(ii) $\int_H Z dP = \int_H X dP \quad \forall H \in \sigma(Y)$

Existence of such a thing can be proven using Radon-Nikodym concepts. Now, $E[X|Y=y]$ can be defined as the value $Z(\omega)$ for any value $\omega$ in which $Y(\omega) = y$.

### Intuitive construction with $Z=E[X|Y]$.

Fix $\delta>0$ with $\delta \approx 0$. Define $H = \{\omega : Y(\omega) \in [y, y+\delta]\}$. Suppose that $Z(\omega) \approx f(y)=E[X|Y=y]$ for almost all $\omega \in H$. Then: $$\int_H X dP = \int_H Z dP \approx \int_H f(y) dP = f(y)P[Y\in [y, y+\delta]]$$ So: $$f(y) \approx \frac{\int_{Y \in [y, y+\delta]} X dP}{P[Y \in [y, y+\delta]]}$$ and so we have an (unrigorous) property that: $$f(y) = \lim_{\delta\rightarrow 0^+} \frac{\int_{Y \in [y, y+\delta]} X dP}{P[Y \in [y, y+\delta]]}$$ There might be some crazy examples where the limit does not exist or does not give the desired result.

### Not useful:

$$\sigma(Y=y) = \{\{Y=y\}, \{Y\neq y\}, S, \phi\}$$

We can formally define $E[X|\sigma(Y=y)]$, but this has nothing to do with $E[X|Y=y]$. It can be shown that, with prob 1, $E[X|\sigma(Y=y)]=E[X]$. That is because, for any event $H \in \{\{Y=y\}, \{Y\neq y\}, S, \phi\}$ we have: \begin{align} \int_H E[X] dP = E[X]\int_H dP = E[X]P[H] &= \left\{ \begin{array}{ll} E[X] &\mbox{ if $H=\{Y\neq y\}$ or $H=S$} \\ 0 & \mbox{ otherwise} \end{array} \right.\\ \int_H X dP &=\left\{ \begin{array}{ll} E[X] &\mbox{ if $H=\{Y\neq y\}$ or $H=S$} \\ 0 & \mbox{ otherwise} \end{array} \right. \end{align}

• Thanks, this makes sense. The only remaining question for me is what does $E[X|\sigma(Y)] (=E[X|(Y)])$ mean then/why is it written like that? Particularly when contrasted with the very similar notation in your 'not useful' section. If $E[X|\sigma(Y)]$ doesn't mean a function of the specific sigma algebra $\sigma(Y=y)$ as you might expect from that form from intro. prob. theory, might the '$|\sigma(Y)$' bit more loosely mean your eq (ii) is 'defined on/valid for domains in $\sigma(Y)$'? Such that it means 'given that you are integrating over a set that is in the (generic) sigma field of Y?' – user3353819 Jul 24 '16 at 1:00
• I'm not sure I understand your comment. In the "useful" case, equation (ii) holds for all sets $H$ in the infinite-collection $\sigma(Y)$. In the "not useful" case, equation (ii) holds for all sets $H$ in the 4-element collection $\sigma(Y=y)$, that is, in the collection $\{\{Y=y\}, \{Y\neq y\}, S, \phi\}$. Considering the infinite number of sets in $\sigma(Y)$ allows you to perform your thought experiment of "shrinking" positive probability sets $B_{\delta}$ down to the zero-probability set $\{Y=y\}$, whereas this limiting intuition is impossible for the 4-element set $\sigma(Y=y)$. – Michael Jul 24 '16 at 18:25
• Another way to view $E[X|Y]$ is as a random variable that "begs" to be integrated, so $E[E[X|Y]] = E[X]$ holds. Similarly, perhaps you can think of $E[X|Y=y]$ as something that is not quite formally defined on its own. It is only defined in terms of the full random variable $Y$ and over all $y \in \mathbb{R}$, and begs to be integrated so that $E[X]=\int_y E[X|Y=y]f_Y(y)dy$ holds. – Michael Jul 24 '16 at 18:26