Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Are the following two the same:




Where $X$ is Markov chain

$X_{t_k} \in \mathcal{R}^n$

$V: \mathcal{R}^n \rightarrow \mathcal{R}$

$g: \mathcal{R}^n \rightarrow \mathcal{R}$

V is some value function.

If not, what is the difference ? Any help would be really appreciated, even in reframing the question


share|cite|improve this question
What are $t_k$, $t_{k+1}$ and how are they linked? – Davide Giraudo Jun 6 '12 at 11:50
$X$ is a stochastic path, which takes value $X_{t_k}$ at time $t_k$ and $X_{t_{k+1}}$ at $t_{k+1}.$ The events happen only at discrete time $\ldots,t_k,t_{k+1},\ldots$ – Shash Jun 6 '12 at 11:57
No, the first is $V(X_{t_{k+1}})$. The second is a function of $X_{t_k}$. If your process was random walk, with $t_k = k$ the first would be $S_{k+1}$ and the second $S_k$.( and $V(S) = S$) – mike Jun 6 '12 at 11:58
I changed the notation now. – Shash Jun 6 '12 at 12:03
The counterexample should still work, by allowing $g(x_1, \ldots, x_n) = x_1$, and $V(X_{t_{k+1}})$ to be a random walk in the first coordinate (or something similar). @mike's comment should then apply. – John Engbers Jun 6 '12 at 14:41

The first random variable carries more information that the second.

To see this, note that your setting is equivalent to the following: three random variables $U$, $W$ and $Z$, defined on the same probability space, $U$ integrable, are given and one is interested in $$ R=\mathrm E(U\mid W,Z),\qquad T=\mathrm E(\mathrm E(U\mid W)\mid Z). $$ Obviously, $R$ determines $T$ since $T=\mathrm E(\mathrm E(R\mid W)\mid Z)$, but there is no way to reconstruct $R$ from $T$.

Consider for example the case where $U=W+Z$ with $W$ and $Z$ independent and integrable. Then, $R=U$ and $T=\mathrm E(U)$.

Note: In your setting, $U=V(X_{t_{k+1}})$, $W=g(X_{t_{k+1}})$ and $Z=X_{t_k}$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.