1
$\begingroup$

Jones has been playing daily games for a very long time. If Jones wins a game, then he wins the next one with probability 0.6; if he has lost the last game but won the one preceding it, then he wins the next with probability 0.7; if he has lost the last 2 games, then he wins the next with probability 0.2.Today is Monday and Jones has just won his game. Find the probability he won yesterday’s game.

My attempt: I set up the Markov chain like below:

\begin{bmatrix}0&0.3&0&0.7\\0&0.8&0&0.2\\0.4&0&0.6&0\\0.4&0&0.6&0\end{bmatrix}

Where state 0,1,2,3 are (win,lose),(lose,lose),(win,win) and (lose,win). Then I calculated the stationary distribution which gave me- (𝜋0,𝜋1,𝜋2,𝜋3)= (0.2,0.3,0.3,0.2).

Then I checked if this Markov chain is time reversible, but it is not. I dont know what to do from here.

$\endgroup$
6
  • $\begingroup$ The way you've stated the problem is a little confusing. Could you state what the states of your Markov chain are. Also, why do you need reversibility? $\endgroup$ Commented Aug 17, 2022 at 17:31
  • $\begingroup$ The states are the result of the last two games since the win/lose in the next game depends on the result of the previous two games. About reversibility, I am not quite sure. Since the question is asking to find the probability of winning the game one day before (rather than in future) based on the present state, I thought I might wanna use reversibility. $\endgroup$
    – Krypt
    Commented Aug 17, 2022 at 17:37
  • $\begingroup$ I don't understand why you have 4 states. It looks like you should have three only: the outcome of the current game, the outcome of the previous game, and the outcome of the game before the previous one. $\endgroup$ Commented Aug 17, 2022 at 17:52
  • $\begingroup$ okay, supposing I set up the Markov chain with three states, what should I do to get the probability of winning the game in the past day? $\endgroup$
    – Krypt
    Commented Aug 17, 2022 at 19:40
  • $\begingroup$ I think you sent up the transition matrix $P$ just fine. Well done! $\endgroup$
    – user801306
    Commented Aug 18, 2022 at 11:27

1 Answer 1

1
$\begingroup$

$\newcommand{\prob}{\mathrm{P}}\newcommand{\R}{{\rm I\!R}}$Essentially you have a Markov chain, where you know $$ \prob[x_{t+1} = j | x_t = i] = p_{ij}, $$ but you are looking for $\prob[x_t = i | x_{t+1} = j]$. Using Baye's rule $$ \prob[x_t = i | x_{t+1} = j] = \frac{\prob[x_{t+1} = j | x_t = i] \prob[x_t = i]}{\prob[x_{t+1} = j]} = \frac{p_{ij}\prob[x_t = i]}{\prob[x_{t+1} = j]}. $$ Now the quantities $\prob[x_t = i]$ and $\prob[x_{t+1} = j]$ depend on the initial distribution and on $t$. Can you take it from there?

If the initial distribution is $v\in\R^3$, then the unconditional probability at time $t$ is $p_t = (P^t)^\top v$ and $\prob[x_{t} = i] = (p_t)_i$ (the $i$-th element of $p_t$), or $\prob[x_{t} = i] = e_i^\top (P^t)^\top v$ so, overall, $$\prob[x_t = i | x_{t+1} = j] = p_{ij}\frac{e_i^\top (P^t)^\top v }{e_j^\top (P^{t+1})^\top v}$$

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .