# Bayes' rule in a Markov chain

Let $X$ be a sequence of discrete values from a finite set $V$. Let $A$ be the transition matrix computed by counting instances of $V_i \rightarrow V_j$ from the sequence $X$. Hence, $P(X_{n+1} = V_i | X_n = V_j) = A(i,j)$.

Now, let the events $A = [X_{n+1} = V_i]$ and $B = [X_n = V_j]$. My question is as follows: is $P(A) = P(B)$?

-
What do you mean by $P(X_{n+1} = V_i)$? This is either zero or one. Ditto for the conditional probabilities. – Yuval Filmus May 30 '11 at 23:31
It's the probability that the current event is $V_i$ --- it's not binary since it depends on the $X$ – Jacob May 30 '11 at 23:34
Jacob: for any events A and B of positive probability, what is the definition of P(A|B)? And of P(B|A)? Ergo? – Did May 31 '11 at 5:55
@Didier: I'm sorry, I don't understand what you're getting at. I've updated my question to make it clearer. – Jacob May 31 '11 at 12:08
Forget the update. Quote: My question is as follows: is P(A|B)=P(B|A) since [meaning: if one assumes that] P(A)=P(B). The answer follows from the very definition of P(A|B) and P(B|A), so I am trying to understand what prevents you from seeing it. Once again: if I give you two events A and B, how do you compute P(A|B)? – Did May 31 '11 at 12:40

The condition that $P(X_n=V_i)=P(X_{n+1}=V_j)$ for every $n$ and every $(i,j)$ is equivalent to the condition that the distribution of $X_n$ is uniform for every $n$, that is, that $P(X_n=V_i)=1/K$ for every $n$ and $i$, where $K$ denotes the size of $V$.

Now, this happens if and only if (1) the distribution of $X_0$ is uniform and (2) the transition matrix $A$ preserves the uniform distribution. Condition (1) means that, for every $i$, $$P(X_0=V_i)=1/K.$$ Condition (2) means that, for every $i$, $$\sum_jA(i,j)=1.$$ Note that the dual condition $\displaystyle\sum_iA(i,j)=1$ is always true.

-

If your transition matrix is $\pmatrix{1&1\cr0&0\cr}$, so that $X_{n+1}=V_1$ no matter what $X_n$ is, then the probability that $X_{n+1}=V_1$ is $1$, while the probability that $X_n=V_2$ is zero, provided $n\gt1$, so it would appear that in this case $P(A)\ne P(B)$.

-
If we can assume homogeneity/ergodicity/whatever property is that means that probabilities are independent of $n$, and we assume we have reached an equilibrium state where each state is occupied at time $n$ according to its equilibrium probability (sorry, don't really know the terminology of Markov chains...) then can we make the statement that for $m\neq n$ we have $P(X_n=i)=P(X_m=j)$ if $i=j$, but otherwise they are not in general equal. – Chris Taylor Jun 2 '11 at 14:41

Reversibility is not required. Let's consider a Markov model of a swimmer. The state WEAR_TRUNKS may be followed by ENTER_WATER (or TAKE_SHOWER). Rarely does one WEAR_TRUNKS after ENTER_WATER.

Look up detailed balance.

-
No, that is not my question. I've updated it to make it clearer. – Jacob May 31 '11 at 0:11