Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

The book by Durrett "Essentials on Stochastic Processes" states on page 55 that:

If the state space S is finite then there is at least on stationary distribution.

  1. How can I find the stationary distribution for example for the square 2x2 matrix $[[a,b],[1-a, 1-b]]$? I have tested this with WA and chains of such form seems to converge to certain probalities, as here. But if you look in the general case, here, I feel quite confused to find the general formula. I just know that it exist but I cannot see any general formula as $n \rightarrow \infty$.

  2. But look here, $[[0,1],[1,0]]$ does not have a stationary distribution! So am I right to say that this chain is depended on initial conditions? If the $M=[[0,1],[1,0]]$, the chain will not converge to stationary condition. But how can I find out this from the matrix? (not through series of observations)

  3. And how can I know when a certain markov chain is depended on initial conditions? For example, with the above example, its $det = a-b$ and for eigenvalues $\lambda_{1} =1$ and $\lambda_{2} = a-b$.

  4. Now Hypertextbook mentions that "behavior, which exhibits sensitive dependence on initial conditions, is said to be chaotic", look we found a case with initial condition sensitivity. Is it chaotic?

share|improve this question
2  
Regarding item 2, for $P=[[0,1],[1,0]]$ vector $\pi = [ \frac{1}{2}, \frac{1}{2} ]$ is definitely the invariant vector, as $\pi^\top P = \pi^\top$. –  Sasha Sep 25 '11 at 20:26

2 Answers 2

up vote 1 down vote accepted

I am not a specialist in invariant distributions but so I hope there will be other answers - just it's to much to write as a comment.

  1. As Sasha wrote you, in there exists an invariant measure $\pi = [0.5,0.5]$ and you can easily check that it works for all $0\leq a,b\leq1$.

  2. The chain will not converge to the stationary distribution since it is periodic with a period $2$. Although the chain does not converge to the stationary distribution, still it exists (since the convergence is sufficient, not necessary condition). You may want to take a look at the notion of ergodicity here.

  3. The chain will depend on the initial condition, say in the case $a=1,b=0$ - then you have to absorbing states, and wherever you start, you stay there forever.

  4. Finally, chaos has no a strict definition (which will order it). To be precise, that statement from the Hypertextbook is not a definition of the chaos. You may say that the chain I told in 3. exhibits the dependence on the initial data, though you may think that non-ergodic chains are chaotic since they exhibit the dependence on the initial distribution.

share|improve this answer

(1) One almost never looks for stationary distributions $\pi$ by looking at powers of $P$. Rather one solves directly the vector equation $P\pi=\pi$, that is, $\pi_x=\sum\limits_yP(x,y)\pi_y$ for every $x$.

(2) This is an example of a periodic chain. This means that you can partition $S$ into sets $S_k$ with $1\le k\le d$ for a given $d$, such that if the chain is in $S_k$ at time $t$, then almost surely the chain is in $S_{k+1}$ at time $t+1$ (and $S_{d+1}=S_1$). The largest possible $d$ is called te period (in your case $d=2$).

You can picture the sets $S_k$ as some points on a circle, then the chain moves along the circle deterministically, for example clockwise from one point to the next one on the circle, and the only stochastic part, if any, is that the chain may visit any state in $S_k$ when time comes to be in $S_k$. If $d=1$ the chain is aperiodic.

And @Sasha already explained that every finite Markov chain, even the periodic ones, has at least one stationary distribution.

(3) and (4) make no sense to me.

You could try reading this or the quite accessible book Markov chains by James Norris.

share|improve this answer
    
I remember seeing a slew of odd downvotes (including on my post here), and my best guess is that someone went on an indiscriminate downvoting spree, targeting whatever posts were on the front page at the time. There's no way I know of to track or undo this, unfortunately. –  Zev Chonoles Sep 27 '11 at 4:51
    
@Zev, OK, right. Another recent one. –  Did Sep 27 '11 at 15:58

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.