3
$\begingroup$

Let $S$ be a countable set and $\pi$ a probability distribution on $S$. A discrete-time Markov chain $(X_n)$ with state space $S$ is said to be in detailed balance with respect to $\pi$ (or simply in detailed balance) if for all states $x$ and $y$, $$\pi(x)P(x \to y) = \pi(y)P(y \to x).$$ (a) Show that if $(X_n)$ satisfies detailed balance, then $\pi$ is a stationary distribution for $(X_n)$.

(b) Consider the general two-state chain with $P(0 \to 1) = p$ and $P(1 \to 0) = q$, where $p,q > 0$. Let $\pi$ be the (unique) stationary distribution. Show the two-state chain always satisfies detailed balance with respect to $\pi$.

(c) Find an irreducible 3-state chain that does not satisfy detailed balance.

(d) Show that any irreducible, positive-recurrent birth-death process satisfies detailed balance with respect to its (unique) stationary distribution.

$\endgroup$
3
  • 1
    $\begingroup$ What did you try? $\endgroup$
    – Davide Giraudo
    Sep 20, 2012 at 10:54
  • 2
    $\begingroup$ Hi, I'm new in this field as well in this site so sorry for inappropriate problem statement. For $(a)$ I'm thinking to sum over x to get the result $\pi p = \pi$ $$\sum_x\pi(x)P(x \to y) = \sum_x\pi(y)P(y \to x) = \pi(y)\sum_xP(y \to x)=\pi(y).$$ Not sure that this is the good way. For $(b)$ I think the solution could be: If $\pi$ is stationary distribution then \begin{cases} \pi(0)P(0 \to 0)+\pi(1)P(1 \to 0)=\pi(0) \ \ \ (1)\\ \pi(0)P(0 \to 1)+\pi(1)P(1 \to 1)=\pi(1) \ \ \ \ (2)\\ \end{cases} $\endgroup$
    – tadas
    Sep 20, 2012 at 11:41
  • $\begingroup$ from $(1)$ we get $$\pi(0)(1-P(0 \to 0))=\pi(0)P(0 \to 1)=\pi(1)P(1 \to 0)$$ from $(2)$ we get $$\pi(1)(1-P(1 \to 1))=\pi(1)P(1 \to 0)=\pi(0)P(0 \to 1)$$ and I would say it' done. Or not? Have no ideas for $(c)$ and $(d)$ $\endgroup$
    – tadas
    Sep 20, 2012 at 11:42

1 Answer 1

Reset to default
2
$\begingroup$

Your answers to parts and (a) and (b) appear correct to me, so I will focus on parts (c) and (d).

For part (c), consider the Markov chain with transition matrix,

$$ P = \begin{pmatrix} 0 & 1 & 0\\ 0 & 0 & 1\\ 1 & 0 & 0 \end{pmatrix}. $$

Then the stationary distribution is given by $\pi=(1/3,1/3,1/3)$, which can be checked by a direct multiplication $\pi P = \pi$. However, the detailed balance equations do not hold.

For part (d), the answer is somewhat heavy in terms of notation, but I will do what I can. Suppose, that you have a birth-death process with birth rates $\{\lambda_i\}_{i\geq0}$ and death rates $\{\mu_i\}_{i\geq1}$. Because the birth-death process is assumed to be positive recurrent, the stationary distribution exists and has the following form.

$$ \pi_n=\frac{1}{c} \frac{\prod_{i=0}^{n-1} \lambda_i}{\prod_{i=1}^{n} \mu_i} $$

The constant $c$ is given by

$$ c=\sum_{n=0}^\infty \frac{\prod_{i=0}^{n-1} \lambda_i}{\prod_{i=1}^{n} \mu_i} < +\infty. $$

The summation is finite by the assumption of positive recurrence. Using these expressions, you can check that the detailed balance equations will hold.

I did not include a derivation of the stationary distribution because your question did not ask for it, but I can provide that as well if you want. They are derived by solving the detailed balance equations for $\pi$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .