Let $X$ be a Markov chain with state space $\mathcal{S}$ and denote $\mathbb{N} := \{ 0, 1, \cdots\}$. We know that for any stopping time $\tau < \infty$ and any bounded measurable function $\phi : \mathcal{S} \longrightarrow \mathbb{R}$, there exists a function $\psi : \mathbb{N}\times \mathcal{S} \longrightarrow \mathbb{R}$ such that

\begin{align} \mathbb{E}[\phi(X_{\tau+1}) \; | \; \mathcal{F}_{\tau}] = \psi(\tau, X_{\tau}) \end{align}

I need to show that $X$ is temporally homogeneous if and only if, for all $\tau$ and $\phi$, the above $\psi$ does not depend on $\tau$ (namely it takes the form $\psi(X_{\tau})$).

I am trying to use smoothing property of conditional expectation to solve this:

\begin{align} \mathbb{1}_{\{\tau = n\}} \; \mathbb{E}[\phi(X_{n+1}) \; | \; \mathcal{F}_n] &= \mathbb{1}_{\{\tau = n\}} \; \mathbb{E}[\phi(X_{n+1}) \; | \; X_n] \\ &= \mathbb{E}[\phi(X_{\tau+1}) \; | \; X_{\tau}] \\ &= \psi(X_{\tau}) \end{align}

I cannot figure out if I have implicitly used time homogeneity here. I also don't know how to establish the sufficiency (how time-homogeneity results from sole dependence of $\psi$ on $X_{\tau}$). I would appreciate any insights on this.

  • $\begingroup$ Why repost an exact copy of a question you already asked (and that people commented)? $\endgroup$
    – Did
    May 6, 2016 at 17:51
  • 1
    $\begingroup$ @user2348674: I also noticed that your acceptance rating is zero :-( You won't get as much attention if you do not ever accept answers. $\endgroup$
    – parsiad
    May 6, 2016 at 17:54
  • $\begingroup$ @Did My previous questionlink was on the proof of strong Markov property whereas this question is on time-homogenous implications of Strong Markov property. $\endgroup$ May 6, 2016 at 18:00
  • $\begingroup$ @par Thanks for pointing that out; I am relatively new here and did not know that I need to do that. Thanks :) ! $\endgroup$ May 6, 2016 at 18:02
  • $\begingroup$ @user2348674: you should go back and accept some answers; it will help your cause. $\endgroup$
    – parsiad
    May 6, 2016 at 18:03

1 Answer 1


So, because $X$ is a Markov chain, there is a function $p:\Bbb N\times S\times S\to[0,1]$ such that $\Bbb P[X_{n+1}=j\,|\,\mathcal F_n] = p(n,i,j)$, a.s. on $\{X_n=i\}$, for each $(n,i,j)\in \Bbb N\times S\times S$. And $X$ is temporally homogeneous if and only if $p(n,i,j) = p(m,i,j)=p(i,j)$ for all $(i,j)\in S\times S$, for all $m$ and $n$.

Added: This answers "I need to show that $X$ is temporally homogeneous if and only if, for all $\tau$ and $\phi$, the above $\psi$ does not depend on $\tau$ (namely it takes the form $\psi(X_{\tau})$)." If $X$ is temporally homogeneous, then $\Bbb E[\phi(X_{n+1}\,|\,\mathcal F_n]=\sum_{j\in S}p(X_n,j)=:\psi(X_n)$ doesn't depend on $n$. From this it follows by conditioning on the value of $\tau$ that $\Bbb E[\phi(X_{\tau+1}\,|\,\mathcal F_\tau]=\psi(X_\tau)$.

Conversely, if $\Bbb E[\phi(X_{\tau+1}\,|\,\mathcal F_\tau]=\psi(X_\tau)$ for all $\phi$ and $\tau$, then taking $\phi=1_{\{j\}}$ and $\tau=n$ (non-random) we have $\Bbb P[X_{n+1}=j\,|\,\mathcal F_n]=\psi_j(X_n)$ for some function $\psi_j$ with values in $[0,1]$. Consequently, $\Bbb P[X_{n+1}=j\,|\,X_n=i]=\psi_j(i)$. Define $p(i,j):=\psi_j(i)$ and note that $\sum_jp(i,j)=1$. It follows that $X$ is temporally homogeneous.

  • $\begingroup$ Is this an answer to the original question I posted? Seems to me this is only the definition of temporal homogeneity of a Markov chain .... $\endgroup$ May 8, 2016 at 1:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .