1
$\begingroup$

There is a system which follows the equation: $X(n+1) = X(n) - 1 + A(n)$, where $A(n)$ is a random variable taking values $0,1,2$ with probabilities $p_0$, $p_1$ and $p_2$, respectively. Now, $X(n) = n$ and $X(n)\geq1$ are two provided conditions. I need to find the relation between $p_0$, $p_1$ and $p_2$ which ensures stability of the Markov Chain.

Answer is $p_0>p_2$, but I can't figure out the way to proceed.

Just for reference, this is actual a model of packet arrivals in a queueing network. Basically, $n$ is an index of discrete time, and $X(n) = n$ implies that the number of packets that arrived at time $n$ equals $n$ itself.

My progress: I tried to derive the steady state matrix, but am not getting $p_0>p_2$ as the answer. In fact, I don't understand how an inequality like this can be obtained.

$\endgroup$
1
$\begingroup$

The transition probabilities are given by $$ P_{ij} =\begin{cases} p_0+p_1,& i=j=0\\ p_0,& j=i-1\\ p_1,& j=i\\ p_2,& j=i+1\\ 0,& \text{ otherwise}.\end{cases} $$ For $n=0,1,2\ldots$ let $$\nu(j) = \left(\frac{p_2}{p_0}\right)^j.$$ Then $\nu=\nu P$, so $\nu$ is an invariant measure for $P$. The system is stable if and only if $$\sum_{j=0}^\infty \nu(j)<\infty,$$ or $p_2<p_0$. In this case, $$\sum_{j=0}^\infty \nu(j) = \sum_{j=0}^\infty \left(\frac{p_2}{p_0}\right)^j = \frac{p_0}{p_0-p_2}.$$ Let $\pi = \left(1-\frac{p_2}{p_0}\right)\nu$, then $\sum_{j=0}^\infty \pi(j)=1$ and $\pi=\pi P$, so $\pi$ is the (unique) stationary distribution for $P$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.