3
$\begingroup$

Let $S$ be a countable set and $\pi$ a probability distribution on $S$. A discrete-time Markov chain $(X_n)$ with state space $S$ is said to be in detailed balance with respect to $\pi$ (or simply in detailed balance) if for all states $x$ and $y$, $$\pi(x)P(x \to y) = \pi(y)P(y \to x).$$ (a) Show that if $(X_n)$ satisfies detailed balance, then $\pi$ is a stationary distribution for $(X_n)$.

(b) Consider the general two-state chain with $P(0 \to 1) = p$ and $P(1 \to 0) = q$, where $p,q > 0$. Let $\pi$ be the (unique) stationary distribution. Show the two-state chain always satisfies detailed balance with respect to $\pi$.

(c) Find an irreducible 3-state chain that does not satisfy detailed balance.

(d) Show that any irreducible, positive-recurrent birth-death process satisfies detailed balance with respect to its (unique) stationary distribution.

$\endgroup$
3
  • 1
    $\begingroup$ What did you try? $\endgroup$ Sep 20 '12 at 10:54
  • 2
    $\begingroup$ Hi, I'm new in this field as well in this site so sorry for inappropriate problem statement. For $(a)$ I'm thinking to sum over x to get the result $\pi p = \pi$ $$\sum_x\pi(x)P(x \to y) = \sum_x\pi(y)P(y \to x) = \pi(y)\sum_xP(y \to x)=\pi(y).$$ Not sure that this is the good way. For $(b)$ I think the solution could be: If $\pi$ is stationary distribution then \begin{cases} \pi(0)P(0 \to 0)+\pi(1)P(1 \to 0)=\pi(0) \ \ \ (1)\\ \pi(0)P(0 \to 1)+\pi(1)P(1 \to 1)=\pi(1) \ \ \ \ (2)\\ \end{cases} $\endgroup$
    – tadas
    Sep 20 '12 at 11:41
  • $\begingroup$ from $(1)$ we get $$\pi(0)(1-P(0 \to 0))=\pi(0)P(0 \to 1)=\pi(1)P(1 \to 0)$$ from $(2)$ we get $$\pi(1)(1-P(1 \to 1))=\pi(1)P(1 \to 0)=\pi(0)P(0 \to 1)$$ and I would say it' done. Or not? Have no ideas for $(c)$ and $(d)$ $\endgroup$
    – tadas
    Sep 20 '12 at 11:42
2
$\begingroup$

Your answers to parts and (a) and (b) appear correct to me, so I will focus on parts (c) and (d).

For part (c), consider the Markov chain with transition matrix,

$$ P = \begin{pmatrix} 0 & 1 & 0\\ 0 & 0 & 1\\ 1 & 0 & 0 \end{pmatrix}. $$

Then the stationary distribution is given by $\pi=(1/3,1/3,1/3)$, which can be checked by a direct multiplication $\pi P = \pi$. However, the detailed balance equations do not hold.

For part (d), the answer is somewhat heavy in terms of notation, but I will do what I can. Suppose, that you have a birth-death process with birth rates $\{\lambda_i\}_{i\geq0}$ and death rates $\{\mu_i\}_{i\geq1}$. Because the birth-death process is assumed to be positive recurrent, the stationary distribution exists and has the following form.

$$ \pi_n=\frac{1}{c} \frac{\prod_{i=0}^{n-1} \lambda_i}{\prod_{i=1}^{n} \mu_i} $$

The constant $c$ is given by

$$ c=\sum_{n=0}^\infty \frac{\prod_{i=0}^{n-1} \lambda_i}{\prod_{i=1}^{n} \mu_i} < +\infty. $$

The summation is finite by the assumption of positive recurrence. Using these expressions, you can check that the detailed balance equations will hold.

I did not include a derivation of the stationary distribution because your question did not ask for it, but I can provide that as well if you want. They are derived by solving the detailed balance equations for $\pi$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.