# Markov chain long run proportion

You are a coach of a good football team, but the owner is very unforgiving. If you lose three games in a row, you are automatically red. So, whenever you lose two games in a row, you bribe the referees in the next game, ensuring that your team wins it. Otherwise, your team wins any game independently with probability p = 0.8. The season is about to begin.

a) Determine the transition probability matrix of the Markov Chain whose state is the number of consecutive games you have lost before the coming game.

b)Write down an expression for the probability that your team wins game 5 and loses game 10. Do not evaluate.

c)Calculate the proportion of games (in a very long season) your team wins.

d) Calculate the proportion of games (in a very long season) your team wins honestly (i.e., without bribing the referees).

e) A neutral observer, who does not usually follow your team, attends one of your game and see you win. What is the probability that you have bribed the referees? Assume the season started a long time ago.

Hi, my main question is part e. I put up my solution for first few parts. Can you also check if answer is correct. If more detailed is required for qa to d, I'll will add.

a) Markov chain for number of consecutive losses with state 0,1 & 2 with transition matrix P

b) P($X_5 = 0$, ($X_{10} = 1$ or $X_{10} = 2$)) = $(p_{01}^5+P_{02}^5)P_{00}^5$

c)$\pi = \pi P, \pi_2 = 1/31, \pi_1$= $(0.2 - 2/100)1/1.2, \pi_0 = 1 - \pi_1 - \pi_2$

d)New markove chain but with 4 states {0,1 ,2,3} consecutive losses. Since state 3 is reccurent and rest is trainsient states, then $\pi_3 = 1$ and rest equal to zero.

e)$P(X_n =0| X_{n-1} = 2) ?$

• "I put up my solution for first few parts" Sorry but how are these supposed to be solutions to the first questions?
– Did
Commented Apr 7, 2018 at 23:31

Let $X_n$ be the number of games lost before game $n$. Then $\{X_n:n=0,1,\ldots\}$ is a Markov chain on $\{0,1,2\}$ with transition matrix given by $$P=\pmatrix{\frac45&\frac15&0\\\frac45&0&\frac15\\1&0&0}.$$ Note that game $n$ is won when $X_n=0$ and lost when $X_n=1$ or $X_n=2$. Hence \begin{align} \mathbb P(X_5 = 0, X_{10}\ne 0\mid X_0=0) &= \mathbb P(X_{10}\ne 0\mid X_5=0,X_0=0)/ \mathbb P(X_5=0\mid X_0=0)\\ &= \left(\mathbb P(X_{10} = 1\mid X_5=0) + \mathbb P(X_{10}=2\mid X_5=0) \right)/ \mathbb P(X_5=0\mid X_0=0)\\ &= \left(P_{01}^5 + P_{02}^5\right)/P^5_{00}\\ &= \left( 1-P_{00}^5\right)/P^5_{00}\\ &= \left(1-\frac{504}{625}\right)/\left(\frac{504}{625}\right)\\ &= \frac{121}{504}. \end{align}
Since all entries of $P^3$ are positive, $X$ is ergodic, so there exists a unique stationary distribution $\pi$ (which necessarily sums to $1$) satisfying $\pi P = \pi$. Hence \begin{align} \pi_1 &=\frac15\pi_0\\ \pi_2 &=\frac15\pi_1\\ 1&=\pi_0+\pi_1+\pi_2, \end{align} whence $\pi_2=\frac1{25}\pi_0$ and thus $$\pi = \left(\frac{25}{31}, \frac5{31},\frac1{31}\right).$$ The limiting proportion of games your team wins is given by $\pi_0=\frac{25}{31}$. The limiting proportion of games your team wins without bribing the referees is $$\pi_0 - \frac15\pi_1 = \frac{25}{31}-\frac15\left(\frac5{31}\right) =\frac{24}{31}.$$ The limiting probability that, conditioned on a game being won, that the game was won by bribing the referees is given by $$\frac{\pi_2 }{\frac45\pi_0 + \frac45\pi_1+\pi_2} = \frac{\frac1{31}}{\frac45\cdot\frac{25}{31}+\frac45\cdot\frac5{31}+\frac1{31}}= \frac1{25}.$$
• Hi sorry to bump this up, but for the part: "The limiting proportion of games your team wins without bribing the refereees is " why is it equal to $\pi_0 - \frac{1}{5}\pi_1$? Thanks Commented Apr 15, 2018 at 3:56