3
$\begingroup$

I'm aware that a Markov chain with a stationary distribution has an inverse which is a Markov chain. But I suspect this to be false in general. Can someone please provide a counterexample?

$\endgroup$
4
  • $\begingroup$ I've given it some thought and I now believe that it's enough to consider a two-states random walk starting from one state with probability $1$. Then $X_0|X_1$ is distributed as a Dirac delta, while $X_1|X_2$ isn't. This is already enough to conclude that the reverse process doesn't have the Markov property. $\endgroup$
    – No-one
    Jul 27, 2023 at 2:07
  • 1
    $\begingroup$ Can you specify what you mean by the inverse of a markov chain? If $X_t$ is a markov chain, then the process $Y_t = X_{T-t}$ for some $T$ is also a markov chain - but e.g. if $X_t$ is time-homogeneous, this does not mean that $Y_t$ is. $\endgroup$
    – a_student
    Jul 27, 2023 at 17:57
  • $\begingroup$ @a_student By inverse of a Markov chain I mean the process you called $Y_t$. You claim that it is also a Markov chain itself, but I now believe it isn't for the reason I stated in my previous comment. Of course I might be wrong, so if you happen to know a proof of your claim please write it down and I will accept it as an answer to my question. $\endgroup$
    – No-one
    Jul 27, 2023 at 18:13
  • $\begingroup$ Please edit your question and add your comments in it. $\endgroup$ Oct 24, 2023 at 23:31

1 Answer 1

2
$\begingroup$

Let $X_t$ be a Markov chain on a discrete state space

$$\mathbb{P}[X_t = x \vert X_{t+1} = y, X_{t+2} =z] = \frac{\mathbb{P}[X_t = x ,X_{t+1} = y, X_{t+2} =z]}{\mathbb{P}[X_{t+1} = y, X_{t+2} =z]}=\\ \frac{\mathbb{P}[ X_{t+2} =z \vert X_{t+1} = y,X_t = x ]\mathbb{P}[X_{t+1} = y,X_t = x]}{\mathbb{P}[X_{t+1} = y, X_{t+2} =z]}\\= \frac{\mathbb{P}[ X_{t+2} =z \vert X_{t+1} = y]\mathbb{P}[X_{t+1} = y,X_t = x]}{\mathbb{P}[X_{t+1} = y, X_{t+2} =z]} =\frac{\mathbb{P}[X_{t+1}=y,X_t=x]}{\mathbb{P}[X_{t+1}=y]} = \mathbb{P}[X_t = x \vert X_{t+1}=y]$$

like this you get the markov property for the reverse process.

But then even if $X_t$ is time-homogenous, that is $\mathbb{P}[X_{t+1} = y \vert X_t = x] = p(y \vert x)$, then you still have $$\mathbb{P}[X_t = x \vert X_{t+1}=y] = \frac{p(y \vert x) \mathbb{P}[X_t=x]}{\mathbb{P}[X_{t+1}=y]},$$ which can depend on time.

$\endgroup$
3
  • $\begingroup$ Thank you, this makes a lot of sense. This answer then shows that the Markov property is intrinsically time-symmetric. Since to me this is unexpected, I opened another question asking if we can reformulate it in such a way that the symmetry becomes more apparent math.stackexchange.com/questions/4743554/…. $\endgroup$
    – No-one
    Jul 27, 2023 at 19:59
  • $\begingroup$ Also, from the displayed equation I think that we can conclude that the inverse is time-homogenous if and only if the initial distribution of $X_0$ is stationary. Indeed, by taking $x=y$ we get $\mathbb{P}(X_t=x)=c_x^t\mathbb{P}(X_0=x)$ for some constant $c_x$, which must be $<=1$ since otherwise take limit $t\to \infty$. But then, from $1= \sum \mathbb{P}(X_1=x)=\sum c_x \mathbb{P}(X_0=x)$, $1=\sum \mathbb{P}(X_0=x)$ and $c_x\in[0,1]$, we get $c_x=1$ for all $x$. $\endgroup$
    – No-one
    Jul 28, 2023 at 0:20
  • $\begingroup$ (continued) My reasoning assumes $p(x|x)>0$ (aperiodicity should be enough), otherwise it's false. For example a deterministic walk on a two-states graph (at each step switch state with probability $1$) started from one state is an example of a time-homogenous Markov chain whose inverse is still a time-homogeneous Markov chain, but the distribution on $X_0$ is not stationary. $\endgroup$
    – No-one
    Jul 28, 2023 at 3:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .