I am studying the Strong Markov Property in Discrete Time Markov Chains. More precisely, I am trying to understand the proof of the theorem below that is found in chapter 3, section 3 page 51 of D. Kannan's book An Introduction to Stocachastic Processes.

I have doubts in 3 passages of the demonstration. The preliminary notations are that $\{X_n,n\geq 0\}$ is a Markov Chain definite in $(\Omega,\mathscr{A},\mathbb{P})$ and $\mathscr{A}_n=\sigma(X_0,X_1,\ldots, X_n)$.

Theorem 3.3.3 Let $\{X_n,n\geq 0\}$ be a Markov Chain with contable state space $\mathcal{S}=\{s_0,s_1,s_2,\ldots\}$ and $m$-step transition probabilities $p^{m}(\,\cdot \,,\,\cdot\,)$ and $\tau$ be a stopping time relative to $\{X_n,n\geq 0\}$ such that $\tau(\omega)<\infty$ for all $\omega\in\Omega$. Define $Y_n(\omega)=X_{\tau(\omega)+n}(\omega), n\geq 0$. Then $\{Y_n,n\geq 0\}$ is an Markov Chain and, for $0=n_0<n_1<\cdots<n_m$, $x_0,x_1,\ldots,x_m\in\mathcal{S}$, we have $$ \mathbb{P}\big( Y_{n_k}=x_k, 0\leq k\leq m \big)=q_0(x_0)\prod_{k=0}^{m-1}p^{n_{k+1}-n_k}(x_k,x_{k+1}),\quad \tag{*} $$ where $q_0$ is the initial distribuition of $Y_0$.

PROOF. Define the $\mathscr{A}_\tau=\{A\in \mathscr{A}: A\cap \{\omega\in\Omega: \tau(\omega)\leq n\}\in\mathscr{A}_n\}$. Then $\mathscr{A}_\tau$ is a $\sigma$-algebra. For $A\in\mathscr{A}_\tau$ there is an $A_n\in \mathscr{A}_n$ such that $$ A\cap \{\tau=n\}=A_n\cap \{\tau=n\}. $$ Let $\mathscr{B}=\sigma\{Y_n:n\geq 0\}$. If $B=\bigcap_{k=1}^{m}\{Y_{n_k}=x_k\}$ and $B_n=\bigcap^{m}_{k=1}\{X_{n+n_k}=x_k\}$, then $B\in\mathscr{B}$, $B_n\in \sigma(X_{n},X_{n+1},X_{n+2},\ldots)$, and $$ B\cap \{\tau=n\}=B_n\cap \{ \tau =n\}. $$ Now \begin{align} \mathbb{P}(A\cap B) =& \sum_{n\leq 0}\sum_{x_0\in \mathcal{S}} \mathbb{P}(A_n\cap \{\tau=n\}\cap \{X_n=x_0\}\cap B_n) \tag{Eq 1}\\ =& \sum_{n\leq 0}\sum_{x_0\in \mathcal{S}} \mathbb{P}(A_n\cap \{\tau=n\}\cap \{X_n=x_0\}) \mathbb{P}(B_n|\{X_n=x_0\}) \tag{Eq 2}\\ =& \sum_{n\leq 0}\sum_{x_0\in \mathcal{S}} \prod_{m=0}^{m-1}p^{n_{k+1}-n_k}(x_k,x_{k+1}) \mathbb{P}(A\cap \{\tau=n\}\cap \{X_n=x_0\}) \tag{Eq 3}\\ =& \sum_{x_0\in \mathcal{S}} \prod_{m=0}^{m-1}p^{n_{k+1}-n_k}(x_k,x_{k+1}) \mathbb{P}(A\cap \{X_n=x_0\}) \tag{Eq 4}\\ \end{align} Since the event $\{Y_0=x_0\}\in\mathscr{A}_\tau$, now choose $A=\{Y=x_0\}$. This yields ($\ast$) e proves the Markov property of $\{Y_n,n\leq 0\}$.

Question 1. How does the equation (Eq 1) imply the equation (Eq 2)? I have no idea how to get the equation (Eq 2) from the equation (Eq 1).

Question 2. How does the equation (Eq 2) imply the equation (Eq 3)? I think it would be from the definition of $\tau$ and the Markov property. But I have no idea how to put those thoughts into practice.

Question 3. How does the equation (Eq 4) imply the equation (*)?

  • $\begingroup$ First question (the others are similar): $$P(B_n\mid A_n,\tau=n,X_n=x_0)=P(B_n\mid X_n=x_0)$$ Does this ring any bell? $\endgroup$ – Did Feb 22 '18 at 21:23
  • $\begingroup$ @Did This is just my doubt. I can not understand why this equality is true. Could you explain this equality better? It seems to me that my question is reduced in the following question: If $C\in\mathscr{A} _n=\sigma( X_0,\ldots, X_{n-1},X_{n})$ then is it true that $P(B_n\mid C, X_n=x_0)=P(B_n\mid X_n=x_0)$? $\endgroup$ – MathOverview Feb 23 '18 at 12:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.