Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let be a probability distribution on the nonnegative integers such that $\pi_i > 0$ for all i. Write down the transition matrix of an irreducible, aperiodic, recurrent Markov chain on the nonnegative integers that has as its stationary probability distribution.

File can be found here: (#1 on this site)

What I notice is that I am trying to reduce it to a finite case, then go from specific to general. Maybe I can try the geometric distribution but I not really seeing how using the detailed balance equations here. Many Thanks

share|cite|improve this question
Since he requires the chain to be aperiodic, he must mean a discrete time chain, not a continuous time chain. – Byron Schmuland May 5 '12 at 18:01
Can you please draw the transition matrix? Thanks – mary May 5 '12 at 21:25
$P$ is an infinite matrix with identical rows. All of the rows look like $(\pi_1,\pi_2 ,\pi_3,\dots)$. – Byron Schmuland May 5 '12 at 21:45
up vote 2 down vote accepted

The easiest example is to let $P_{ij}=\pi_j$ for all non-negative integers $i,j$. This corresponds to a chain that jumps to state $j$ with probability $\pi_j$ regardless of the initial state. Starting at time 1, the Markov chain $(X_n)_{n\geq 1}$ consists of i.i.d. random variables.

Since $P_{ij}>0$ and $P_{ji}>0$, all states communicate so the chain is irreducible.

Since $P_{ii}>0$ the period of state $i$ is 1, so the chain is aperiodic.

Since $\pi$ is an invariant probability vector, the chain is positive recurrent.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.