# Markov chains example

Your exam could be marked with a range of possible grades, simplified as on the following state diagram:

To begin with the chances are that you will pass with a standard result. Each 45 minutes (the time allocated for each answer) your grade could go up or down to a neighbouring grade or stay put.

There is a chance that your Genius is so great that the examiner cannot appreciate what you are saying and incorrectly marks it as nonsense.

What is your grade most likely to be after the 3 hour exam? What (if any) are the steady state results for a group of 14 students?

OK, I have done following:

 1/2     1/2      0    0
1/10    3/10    3/5    0
0     1/2    2/5 1/10
1/10       0    4/5 1/10


Then computed P^4 since in 3 hours there is 4x45 mins

    7/50     401/1000       21/50     39/1000
11/125   999/2500   1149/2500      33/625
407/5000 779/2000   2383/5000      21/400
417/5000 161/400     459/1000   551/10000


Then if starting from Standard grade, to get grade most likely to be is to look for 3rd row, which leads to that in 3 hours grade student most likely to get is of course Standard, and is around 0.45!

I have completed first task. Second I am unfamiliar, I don't know how to resolve it. What do we require in this process?

Hint: You should fine the stationary distribution (I guess) by solving the system $\pi P=\pi$ i.e. $$(\pi_1,\pi_2,\pi_3,\pi_4)\begin{pmatrix} 1/2 & 1/2 & 0 & 0 \\ 1/10 & 3/10 &3/5&0\\0&1/2&2/5&1/10\\1/10&0&4/5&1/10\end{pmatrix}=(\pi_1,\pi_2,\pi_3,\pi_4)$$ and $\pi_1+\pi_2+\pi_3+\pi_4=1$. Then multiply the stationary probabilities $\pi_i, i=1,2,3,4$ with 14, to find the percentage of students that are in the categories fail, acceptible, standard and genius.
If my calculations are correct then $\pi_1=\frac{8}{63},\pi_2=\frac{25}{63},\pi_3=\frac{27}{63},\pi_4=\frac{3}{63}$, and therefore in a class of $14$ the stationary states will be approximately $1.78$ fails, $6$ acceptible, $5.56$ standard and $0.67$ genius. If we have to round the numbers then it will be $2$ fails, $6$ acceptible, $5$ or $6$ standard and $1$ or $0$ geniuses respectively. (Please doublecheck the calculations because I am very prone to mistakes in matrix calculations).
• You should find the same result. These states are called steady because they are independent of time 1,2,3 or 4. Note that since $$vP=v$$ you have that $(vP)P=v \Rightarrow (vP)P^2=v \Rightarrow (vP)P^3=v$ so $vP^4=v$ which means that the $v$ we found satisfies also $vP^4=v$. Mar 1, 2014 at 0:13