3
$\begingroup$

Harry's restaurant changes year after year between the states $0$ (bankrupt), $1$ (almost bankrupt) and $2$ (solvent).

The transition matrix is $P= \begin{pmatrix} 1 & 0 & 0 \\ 1/2 & 1/4 & 1/4 \\ 1/2 & 1/4 & 1/4 \end{pmatrix}$

Calculate the expected value for the amount of years till state $0$ is reached, if we started from state $2$.

I took this question from an exam and try to solve it but I'm not sure how to do this correct? I'm a bit confused we need to work with expected value to calculate the required steps / years to get from state $2$ to state $0$. At least that's how I understood this so far.

It all sounds like I need to solve some recursive relations. Let $h(k)$ be the expected number of steps / years in this example until we reach the state $0$ when you are in state $2$. So we have that $$h(2)=0$$

because when you are in state $2$, you need $0$ steps / years to reach $2$. Then for $k=1$

$$h(1) = 1+0.25h(1)+0.25h(2)+0.5h(0)$$

because when you are in state $1$ you will need a step ($+1$) so you will reach with probability $0.25$ state $1$ again and with probability $0.25$ state $2$ and with probability $0.5$ state $0$.

Similarly we do this for $h(0):$

$$h(0) = 1+1h(0)$$

But from here I don't really know how to continue to get the system and calculate the expected number pf steps with that? : /

$\endgroup$
  • 1
    $\begingroup$ "Calculate the expected value for the amount of years till state 0 is reached, if we started from state 2" vs "when you are in state 2, you need 0 steps / years to reach 2" Confusing states 2 and 0? $\endgroup$ – Did Feb 3 '18 at 18:25
  • $\begingroup$ @Did Yeah.. I should have better read what I have written before posting it / handing it in next time :p $\endgroup$ – roblind Feb 3 '18 at 20:19
7
$\begingroup$

Let $h(k)$ be the expected time to reach state $0$ if we started from state $\color{blue}k$.

Then $h(0)=0$.

And if we start with state $1$, with probability $\frac12$ we reach state $0$, with probability $\frac14$ we reach state $1$, and with probability $\frac14$ we reach state $2$.

Hence $$h(1)=1+\frac12h(0)+\frac14h(1)+\frac14h(2)$$

Similarly,

$$h(2)=1+\frac12h(0)+\frac14h(1)+\frac14h(2)$$

Substituting $h(0)=0$, we have

$$h(1)=1+\frac14h(1)+\frac14h(2)\tag{1}$$

$$h(2)=1+\frac14h(1)+\frac14h(2)\tag{2}$$

Subtracting both equation we have $$h(1)=h(2)\tag{3}$$

Use equation $(3)$ and $(2)$ to solve for $h(2)$.

$\endgroup$
5
$\begingroup$

This is a Markov chain with an absorbing state. Denote by $Q$ the non-absorbing block of the transition matrix. The fundamental matrix is $N= (I-Q)^{-1} = $

$$ \begin{pmatrix} \frac{3}{4} & -\frac{1}{4} \\ -\frac{1}{4} & \frac{3}{4} \\ \end{pmatrix} ^{-1} = \frac{1}{2} \begin{pmatrix} 1 & 3\\ 3 & 1\\ \end{pmatrix} $$

This matrix's entry $(i, j)$ is the expected value of visits at $i$ before being absorbed if the chain starts at $j$ (or the other way around, I don't remember, but luckily it doesn't matter in this case the matrix is symmetric). So the answer is

$$\frac{1}{2}(3+1) = 2.$$


EDIT

The meanings of the entries of $N$ are actually other way around (since the transition matrix is given such that "rows sum to $1$"). Here's a reference (page 15 of the pdf (or 419 as the page number)).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.