Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Let $S$ be a countable set and $\pi$ a probability distribution on $S$. A discrete-time Markov chain $(X_n)$ with state space $S$ is said to be in detailed balance with respect to $\pi$ (or simply in detailed balance) if for all states $x$ and $y$, $$\pi(x)P(x \to y) = \pi(y)P(y \to x).$$ (a) Show that if $(X_n)$ satisfies detailed balance, then $\pi$ is a stationary distribution for $(X_n)$.

(b) Consider the general two-state chain with $P(0 \to 1) = p$ and $P(1 \to 0) = q$, where $p,q > 0$. Let $\pi$ be the (unique) stationary distribution. Show the two-state chain always satisfies detailed balance with respect to $\pi$.

(c) Find an irreducible 3-state chain that does not satisfy detailed balance.

(d) Show that any irreducible, positive-recurrent birth-death process satisfies detailed balance with respect to its (unique) stationary distribution.

share|improve this question
1  
What did you try? –  Davide Giraudo Sep 20 '12 at 10:54
2  
Hi, I'm new in this field as well in this site so sorry for inappropriate problem statement. For $(a)$ I'm thinking to sum over x to get the result $\pi p = \pi$ $$\sum_x\pi(x)P(x \to y) = \sum_x\pi(y)P(y \to x) = \pi(y)\sum_xP(y \to x)=\pi(y).$$ Not sure that this is the good way. For $(b)$ I think the solution could be: If $\pi$ is stationary distribution then \begin{cases} \pi(0)P(0 \to 0)+\pi(1)P(1 \to 0)=\pi(0) \ \ \ (1)\\ \pi(0)P(0 \to 1)+\pi(1)P(1 \to 1)=\pi(1) \ \ \ \ (2)\\ \end{cases} –  tadas Sep 20 '12 at 11:41
    
from $(1)$ we get $$\pi(0)(1-P(0 \to 0))=\pi(0)P(0 \to 1)=\pi(1)P(1 \to 0)$$ from $(2)$ we get $$\pi(1)(1-P(1 \to 1))=\pi(1)P(1 \to 0)=\pi(0)P(0 \to 1)$$ and I would say it' done. Or not? Have no ideas for $(c)$ and $(d)$ –  tadas Sep 20 '12 at 11:42
add comment

1 Answer

Your answers to parts and (a) and (b) appear correct to me, so I will focus on parts (c) and (d).

For part (c), consider the Markov chain with transition matrix,

$$ P = \begin{pmatrix} 0 & 1 & 0\\ 0 & 0 & 1\\ 1 & 0 & 0 \end{pmatrix}. $$

Then the stationary distribution is given by $\pi=(1/3,1/3,1/3)$, which can be checked by a direct multiplication $\pi P = \pi$. However, the detailed balance equations do not hold.

For part (d), the answer is somewhat heavy in terms of notation, but I will do what I can. Suppose, that you have a birth-death process with birth rates $\{\lambda_i\}_{i\geq0}$ and death rates $\{\mu_i\}_{i\geq1}$. Because the birth-death process is assumed to be positive recurrent, the stationary distribution exists and has the following form.

$$ \pi_n=\frac{1}{c} \frac{\prod_{i=0}^{n-1} \lambda_i}{\prod_{i=1}^{n} \mu_i} $$

The constant $c$ is given by

$$ c=\sum_{n=0}^\infty \frac{\prod_{i=0}^{n-1} \lambda_i}{\prod_{i=1}^{n} \mu_i} < +\infty. $$

The summation is finite by the assumption of positive recurrence. Using these expressions, you can check that the detailed balance equations will hold.

I did not include a derivation of the stationary distribution because your question did not ask for it, but I can provide that as well if you want. They are derived by solving the detailed balance equations for $\pi$.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.