0
$\begingroup$

Limiting distribution
In many cases, a Markov chain exhibits a long-term limiting behavior. The chain settles down to an equilibrium distribution, which is independent of its initial state.

De nition Let $(X_n)_{n \in N_0}$ be a Markov chain with transition matrix $P$. A limiting distribution for the Markov chain is a probability distribution $\lambda$ such that for all $i$ and $j$

$$\lim_{x \to \infty} (P^n)_{ij} = \lambda_j$$

Some examples Let $(X_n)_{n \in N_0}$ be a Markov chain with transition matrix P. Find, if it exists, a limiting distribution for $(X_n)_{n \in N_0}$.

Example
1. $ P = \begin{bmatrix} 1 & 0\\ 0 & 1 \end{bmatrix} $
2. $ P = \begin{bmatrix} 1 & 0\\ 0 & 1 \end{bmatrix} $

If $(X_n)_n$ has a limiting distribution $ \lambda = (\lambda_j)_{j > \in S}$, then ...

  • ... for any initial distribution, and for all $j$, $\lim_{n \to \infty} P(X_n = j) = \lambda_j$,

  • ... for any initial distibution, $ \lim_{n \to \infty} \alpha P^n = \lambda$,

  • ... if $\Lambda$ is a stochastic matrix all of whose rows are equal to \lambda, then $\lim_{n \to \infty} P^n = \Lambda$

Stationary distribution
Let's see what happens if we assign the limiting distribution of a Markov chain to be its initial distribution.

Example
For the two-state chain with

$$P = \begin{bmatrix} 1-p & p\\ q & 1-q \end{bmatrix}$$

the limiting distribution can be shown to be

$$\lambda = (\frac{q}{p+q}, \frac{p}{p+q})$$

Find the distribution of $X_1$.

A probability vector $\pi$ that satis es $\pi = P \pi$ plays a special role in the theory of Markov chains.

De nition
Let $(X_n)_{n \in N_0}$ be a Markov chain with transition matrix $P$. A stationary distribution is a probability distribution $\pi$, which satisfy

$$\pi = P \pi$$

That is, $$\pi_{j} = \sum_{i \in S}^{} \pi_i P_{ij} \text{ } > \forall j \in S$$

I haven't understood the above discussion from my lecture slides.

What I understood from outside this material that there is a special/unique vector, $t$, which satisfies $tP=t$ which is called Stationary Distribution (Seymour Lipchuz, Page-$133$).

But, I am confused about the term Limiting Distribution. It seems to be a matrix.

So, shouldn't it be called Limiting Matrix?.

$\endgroup$
  • $\begingroup$ Comments: Sometimes it is possible to have more than one stationary distribution for example with your example matrices (are they supposed to be the same?), but that makes less sense for limiting distributions. Your $\lambda$ and $\pi$ look more like vectors than matrices. $\endgroup$ – Henry Mar 21 at 8:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.