Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

This question is about finding the limit kernel $P^\infty$ of a non-ergodic Markov Chain.

The problem

We consider a TDMC (Time Discrete Markov Chain) $(X_t)_{t \geq 0}$ with $X \in \mathcal{X}$ (having $\mathcal{X}$ as the set of the chain states) under the assumption that the chain is non-ergodic and that $t \in \mathbb{N}$.

We know that to calculate the state probability vector $\pi(t) = (\pi_1(t), \pi_2(t) \dots \pi_n(t))$, where $\pi_i(t) = P\{ X_t = i \}$, we use the formula: $\pi(t) = \pi(t-1) \cdot P$ as we suppose having $\pi(0)$ as initial state the final formula becomes: $\pi(t) = \pi(0) \cdot P^t$.

Steady state probabilities

We all know that if the chain were ergodic it would be possible to get the steady state probability vector $\pi = \pi(0) \cdot P^\infty$. But instead of using this involving many calculations, a simpler system is used: $\pi = \pi \cdot P$ adding, of course, the probability normalization rule $\sum \pi_i = 1$ as a substitute of one equation in the afore-mentioned system.

In ergodic conditions the final steady state probability vector $\pi$ is always the same regardless the initial condition $\pi(0)$. In fact if the chain is ergodic then we have that $P^\infty$ has the following form:

$$ P^\infty = \begin{pmatrix} p^\infty_{1,1} & p^\infty_{1,2} & \cdots & p^\infty_{1,n} \\ p^\infty_{2,1} & p^\infty_{2,2} & \cdots & p^\infty_{2,n} \\ \vdots & \vdots & \ddots & \vdots \\ p^\infty_{n,1} & p^\infty_{n,2} & \cdots & p^\infty_{n,n} \end{pmatrix} = \begin{pmatrix} p^\infty_1 & p^\infty_2 & \cdots & p^\infty_n \\ p^\infty_1 & p^\infty_2 & \cdots & p^\infty_n \\ \vdots & \vdots & \ddots & \vdots \\ p^\infty_1 & p^\infty_2 & \cdots & p^\infty_n \end{pmatrix}$$

Non ergodicity

If the chain is not ergodic but there are no periodicity, we have that the limit kernel $P^\infty$ is in the form:

$$ P^\infty = \begin{pmatrix} p^\infty_{1,1} & p^\infty_{1,2} & \cdots & p^\infty_{1,n} \\ p^\infty_{2,1} & p^\infty_{2,2} & \cdots & p^\infty_{2,n} \\ \vdots & \vdots & \ddots & \vdots \\ p^\infty_{n,1} & p^\infty_{n,2} & \cdots & p^\infty_{n,n} \end{pmatrix}$$

The question

Well, I would like to find a mathematical way to get the elements of this matrix (non-ergodic case) using a system or some tool without algorithmically performing $P \cdot P \dots P$.

share|improve this question
1  
The probabilty goes to zero for the transient states. For the non recurrent states the limit will be block diagonal with one block for each communicating class. –  mike Apr 23 '12 at 11:42
    
Mike could you please explain me better? –  Andry May 18 '12 at 2:49
    
Check out the 'communicating classes' in your textbook. You want to break the chain into them. If the class is transient, the probability will go to $0$, that is always true. If the class is recurrent, it will be postitive recurrent because of the finiteness of the chain, and restricted to that class the chain is ergodic with a unique stationary distribution that you find in the usual way. –  mike May 18 '12 at 14:38
    
Renumber you states so that recurrent classes occur as a block of states. Then when you iterate the transition matrix , that block will converge to the ! stationary distribution, make up an example in matlab or R and try it. –  mike May 18 '12 at 14:38

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.