1
$\begingroup$

Consider a Markov chain on the nonnegative integers with transition matrix $\mathbf{P}$ given by $P_{i,i+1}=p$ and $P_{i,0}=1-p$ for all $i$, and for some $0<p<1$. Find a stationary distribution $\vec{\pi}$. Is it unique?

I know this is an irreducible transition matrix so it has a unique stationary distribution. I have tried creating a system of equations but that does not get me anywhere and have looked at finding the eigenvectors.

$\endgroup$

1 Answer 1

1
$\begingroup$

The $i$th row of $P$ looks like $\begin{bmatrix} 1-p & 0 & 0 & \dots & p & 0 & \dots & 0 \end{bmatrix}$, where the $p$ is the $i+1$ position. The $i$th column then looks like $\begin{bmatrix} 0 \\ 0 \\ \dots \\ p \\ 0 \\ \dots \\ 0 \end{bmatrix}$ where now the $p$ is in the $i-1$ position, for $i>0$; for $i=0$ all the entries of the column are $1-p$. So the equations for the stationary distribution, for $i>0$, are $\pi_i=p \pi_{i-1}$. This equation is very simple to solve for a solution which involves $\pi_0$. You can then select $\pi_0$ based on the normalization condition to finish the problem.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .