Consider a stochastic process $\{x_t\}_{t\in T}$ adapted to some filtered probability space $(\Omega,\mathcal{F},\{\mathcal{F}\}_{t\in T},\mathbb{P})$ taking values in the state space $(\mathbb{R},\mathcal{B})$

I wish to consider the probability Pr$(x_t\in\mathcal{A}|x_s)$ where $(s<t)$

This should be a sensible question, as I should be able to assign probability to a question of the form "What is the chance I obtain an outcome $x_t \in [0.5,0.6]$ given that last time I got $x_s=0.4$". e.g. a transition probability.

Naturally we have Pr$(A| B)=$Pr$(A\cap B)/$Pr$(B)$ but then we have Pr$(B)=$Pr$(x_s)=0$ for any particular value.

Now I think that this is where we have the notion of regular conditional probability entering which, as I understand it means we write:

$$\text{Pr}(x_t\in \mathcal{A}|x_s=B)=\lim_{\mathcal{B}\to B}\frac{\text{Pr}(x_t\in \mathcal{A}\cap x_s \in \mathcal{B})}{\text{Pr}(x_s\in \mathcal{B})}$$

Vagaries of the meaning of $\lim_{\mathcal{B}\to B}$ aside (which would have to be implementation specific e.g. here $\lim_{r\to 0} \text{Pr}(x_s\in (B-r,B+r)$), is the above correct?

Should I understand this as a Radon Nikodym derivative? It seems related, but not identical.

How do I relate this to, and formulate it in such a way to be consistent to, how conditional probabilities are usually defined? (as I understand it) viz


Surely $$\mathbb{E}_{\mathbb{P}}[1_{\mathcal{A}}(x_t)|\sigma({B})]$$ just wouldn't work? i.e. $\sigma(B)\nsubseteq\mathcal{F}$?

Is it legitimate to construct Radon-Nikodym derivatives out of measures formed from regular conditional probabilities? i.e. is this (heuristically) ok?

$$\frac{d\mathbb{P}(x_t|x_s=B)}{d\mathbb{Q}(x_t|x_s=B)}=\lim_{\mathcal{A}\to\emptyset}\lim_{\mathcal{B}\to B}\frac{\mathbb{P}(x_t\in\mathcal{A}|x_s\in\mathcal{B})}{\mathbb{Q}(x_t\in\mathcal{A}|x_s\in\mathcal{B})}$$



Based on discussion in the comments, the issues appears to boil down to why one can write

$$\text{Pr}(x_t\in \mathcal{A}|B)=\mathbb{E}_{\mathbb{P}}[1_{\mathcal{A}}(x_t)|\sigma({B})]$$

when $B$ is a zero probability event. What I don't understand is what the sigma algebra generated by a zero probability event looks like and why you can condition on it.

Surely the sigma algebra generated by a zero probability event is itself formed from (complements and unions of) zero probability events in $\mathbb{P}$, thus not in $\mathcal{F}$

e.g. $$\sigma(x_s=B)=\{\omega,\omega^c,\emptyset,\Omega\}\nsubseteq\mathcal{F}$$ with $\mathbb{P}(\omega)=0$ such that $\mathbb{E}_{\mathbb{P}}[f(x_t)|\sigma({B})]=\mathbb{E}_{\mathbb{P}}[f(x_t)]$ or $0$?

so why can we condition on these?

What am I getting wrong here?

  • $\begingroup$ It seems the same question could be simplified to just considering a random vector $(X,Y)$, and defining $P[X \in A | Y=y]$, no? The stochastic process and filtration stuff seems like a distraction from the main concepts here. $\endgroup$
    – Michael
    Jul 23 '16 at 14:00
  • $\begingroup$ A non-measure theory version of probability might "define" this by flipping the conditioning, so $P[X\in A| Y=y] = \frac{f_{Y|X \in A}(y)P[X \in A]}{f_Y(y)}$, assuming $f_Y(y)\neq 0$ and $f_{Y|X \in A}(y)$ makes sense. $\endgroup$
    – Michael
    Jul 23 '16 at 14:03
  • $\begingroup$ Ok, so I'm simplifying what I need this for (which ultimately is going to be random functions 'given' other functions), hence the desire for a proper measure theoretic understanding, but was hoping to streamline the discussion with the above. Granted you can do the above with probability densities. $\endgroup$ Jul 23 '16 at 14:16
  • $\begingroup$ If you are comfortable with expectations given sigma algebras, why not just use $P[X \in A|\sigma(Y)] = E[1_A(X)|\sigma(Y)]$? $\endgroup$
    – Michael
    Jul 23 '16 at 14:17
  • $\begingroup$ Perhaps this is where my intuition breaks. Is conditioning on $\sigma(Y=y)$ legitimate and if so, as you suggest, why? I understand conditioning on $\sigma(\mathcal{Y})$ where $Y\in\mathcal{Y}$. $\endgroup$ Jul 23 '16 at 14:23

Just a summary of my comments that may clarify: You "should never" consider the sigma algebra generated by an event (since it has nothing to do with expressions like $E[X|Y=y]$). You "should" consider the sigma algebra generated by a random variable, and this has lots to do with expressions like $E[X|Y=y]$. Specifically...

Let $S$ be a sample space. Let $(X,Y)$ be a random vector that maps $S\rightarrow \mathbb{R}^2$. So $X(\omega)$ and $Y(\omega)$ are real numbers for all $\omega \in S$.


$$\sigma(Y) = \mbox{Sigma algebra generated by random variable $Y$}$$

Then $\sigma(Y)$ has many events, including all events of the form $\{Y\leq y\}$ and $\{Y \in [y, y+\delta]\}$. Now, $E[X|Y]$ (sometimes written $E[X|\sigma(Y)]$) is a random variable with certain properties. There are different "versions," but they all differ only on a set of measure 0. Let $Z=E[X|Y]$ be a particular version. It is "$Y$-measurable" and:

(i) $Z(\omega) = f(Y(\omega))$ for some function $f$ and for all $\omega \in S$.

(ii) $ \int_H Z dP = \int_H X dP \quad \forall H \in \sigma(Y) $

Existence of such a thing can be proven using Radon-Nikodym concepts. Now, $E[X|Y=y]$ can be defined as the value $Z(\omega)$ for any value $\omega$ in which $Y(\omega) = y$.

Intuitive construction with $Z=E[X|Y]$.

Fix $\delta>0$ with $\delta \approx 0$. Define $H = \{\omega : Y(\omega) \in [y, y+\delta]\}$. Suppose that $Z(\omega) \approx f(y)=E[X|Y=y]$ for almost all $\omega \in H$. Then: $$ \int_H X dP = \int_H Z dP \approx \int_H f(y) dP = f(y)P[Y\in [y, y+\delta]] $$ So: $$ f(y) \approx \frac{\int_{Y \in [y, y+\delta]} X dP}{P[Y \in [y, y+\delta]]} $$ and so we have an (unrigorous) property that: $$ f(y) = \lim_{\delta\rightarrow 0^+} \frac{\int_{Y \in [y, y+\delta]} X dP}{P[Y \in [y, y+\delta]]} $$ There might be some crazy examples where the limit does not exist or does not give the desired result.

Not useful:

$$\sigma(Y=y) = \{\{Y=y\}, \{Y\neq y\}, S, \phi\} $$

We can formally define $E[X|\sigma(Y=y)]$, but this has nothing to do with $E[X|Y=y]$. It can be shown that, with prob 1, $E[X|\sigma(Y=y)]=E[X]$. That is because, for any event $H \in \{\{Y=y\}, \{Y\neq y\}, S, \phi\}$ we have: \begin{align} \int_H E[X] dP = E[X]\int_H dP = E[X]P[H] &= \left\{ \begin{array}{ll} E[X] &\mbox{ if $H=\{Y\neq y\}$ or $H=S$} \\ 0 & \mbox{ otherwise} \end{array} \right.\\ \int_H X dP &=\left\{ \begin{array}{ll} E[X] &\mbox{ if $H=\{Y\neq y\}$ or $H=S$} \\ 0 & \mbox{ otherwise} \end{array} \right. \end{align}

  • $\begingroup$ Thanks, this makes sense. The only remaining question for me is what does $E[X|\sigma(Y)] (=E[X|(Y)])$ mean then/why is it written like that? Particularly when contrasted with the very similar notation in your 'not useful' section. If $E[X|\sigma(Y)]$ doesn't mean a function of the specific sigma algebra $\sigma(Y=y)$ as you might expect from that form from intro. prob. theory, might the '$|\sigma(Y)$' bit more loosely mean your eq (ii) is 'defined on/valid for domains in $\sigma(Y)$'? Such that it means 'given that you are integrating over a set that is in the (generic) sigma field of Y?' $\endgroup$ Jul 24 '16 at 1:00
  • $\begingroup$ I'm not sure I understand your comment. In the "useful" case, equation (ii) holds for all sets $H$ in the infinite-collection $\sigma(Y)$. In the "not useful" case, equation (ii) holds for all sets $H$ in the 4-element collection $\sigma(Y=y)$, that is, in the collection $\{\{Y=y\}, \{Y\neq y\}, S, \phi\}$. Considering the infinite number of sets in $\sigma(Y)$ allows you to perform your thought experiment of "shrinking" positive probability sets $B_{\delta}$ down to the zero-probability set $\{Y=y\}$, whereas this limiting intuition is impossible for the 4-element set $\sigma(Y=y)$. $\endgroup$
    – Michael
    Jul 24 '16 at 18:25
  • $\begingroup$ Another way to view $E[X|Y]$ is as a random variable that "begs" to be integrated, so $E[E[X|Y]] = E[X]$ holds. Similarly, perhaps you can think of $E[X|Y=y]$ as something that is not quite formally defined on its own. It is only defined in terms of the full random variable $Y$ and over all $y \in \mathbb{R}$, and begs to be integrated so that $E[X]=\int_y E[X|Y=y]f_Y(y)dy$ holds. $\endgroup$
    – Michael
    Jul 24 '16 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.