Consider the 2D Markov chain $X_n = (Y_n,Z_n)$ with a simple symmetric random walk along $(y,0),y\in \mathbb Z$ and simple symmetric random walks along the vertical direction for every $y \neq 0$.

Now, the projection of $(Y_n,Z_n)$ on $Y_n$ is not a Markov chain.

Let $T_n=\inf\{k> T_{n-1} : Z_k = 0\}$, so the $n$th hitting time of the $Y$-Axis for the process $(Y_j,Z_j)$.

Is $Y_{T_n}$, the process evaluated at those hitting times, a Markov chain?

The idea is that $$\mathbb P(Y_{T_{n+1}} \vert Y_{T_{n}},Y_{T_{n-1}},\dots,Y_{T_{0}}) = P(Y_{T_{n+1}} \vert Y_{T_{n}}),$$

namely for $(a,0),a\neq 0$,
$$\mathbb P(Y_{T_{n+1}} = (a-1,0)\vert Y_{T_{n}}=(a,0)) = \mathbb P(Y_{T_{n+1}} = (a+1,0)\vert Y_{T_{n}}=(a,0)) = 1/4$$


$$\mathbb P(Y_{T_{n+1}} = (a,0)\vert Y_{T_{n}}=(a,0)) = 1/2,$$

since for the next step in the vertical direction (each with probability $1/4$), the next time the process hits the axis is surely again $(a,0)$.

Is this construction correct?

  • $\begingroup$ Yes. $ $ $ $ $ $ $\endgroup$ – Did Apr 3 '13 at 11:44
  • $\begingroup$ What happens to this question? Is it dead? If so, close it. $\endgroup$ – Did Jun 29 '13 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.