1
$\begingroup$

Imagine I have a discrete-time, discrete-space Markov chain with some transition matrix $B$ and stationary distribution $\pi$. If I know what state I'm in at some time $t$, how can I calculate $p ( X_{t-1} \mid X_t)$, the probability that I came from each possible state?

To give concreteness, let's say:

$$ B = \begin{bmatrix} 0.6 & 0.2 & 0.1 & 0.1 \\ 0.2 & 0.5 & 0.2 & 0.1 \\ 0.1 & 0.1 & 0.5 & 0.3 \\ 0.1 & 0.2 & 0.3 & 0.4 \end{bmatrix} $$ $$ \pi = \begin{bmatrix} 0.2491 & 0.2454 & 0.2821 & 0.2234 \end{bmatrix} $$ $$ X_t = \begin{bmatrix} 0 & 1 & 0 & 0 \end{bmatrix} $$

How do I calculate $p(X_{t-1}\mid X_t)$?

$\endgroup$
  • $\begingroup$ Can we do something like this: If the transition matrix is invertible, we can calculate $p{X_{t-1}}$ from $p_{X_t}$ since $p_{X_t} = p_{X_{t-1}}B$? Then we can use the Bayes rule $\endgroup$ – M. T Sep 28 '16 at 2:57
  • $\begingroup$ Is your transition matrix invertible? If so then its inverse should give the desired quantity... $\endgroup$ – Ian Sep 28 '16 at 3:05
0
$\begingroup$

According to me you are asked to find the transition probabilities of the reversed Markov chain.

Let us denote the Markov chain that you have by $(X_t, t\ge0)$. Its transition probability matrix is $B=(b_{ij})$ i.e. $b_{ij}=P(X_t=j|X_{t-1}=i)$ for any $t$. You are also given the steady state distribution $\vec \pi=(\pi_1,\pi_2,...)$ of the chain $(X_t, t\ge0)$.

The Markov property is stated as “the future is independent of the past given the present state”. Is can be restated as “the past is independent of the future given the present state”. But this means that the process in reverse time, is still a Markov chain. So for your Markov chain $(X_t, t\ge0)$ there exists a reversed version Markov chain, say $(X^*_t, t\ge0)$, which goes back in time. This argument can be made more clear and strict but it is not the proper place here to do it (check the chapters on Reversible Markov chain in the classic books on queueing theory and markov chains. For example, Stochastic Modeling and the Theory of Queues by Ronald W. Wolff).

Denote the transition probabilities of $(X^*_t, t\ge0)$ by $b^*_{ij}$. So what you are asking, is how to compute $b^*_{ij}$.

The theory says that $b^*_{ij}$ can be exactly computed in terms of $b_{ij}$ and $\pi$ in the following way: $$ b^*_{ij}={\pi_j \over \pi_i}b_{ji}. $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.