0
$\begingroup$

Given is a markov chain with transition matrix

$M=\begin{pmatrix} 0.5 & 0.5 & 0 & 0 & 0 & 0\\ 0.25 & 0.75 & 0 & 0 & 0 & 0\\ 0.25 & 0.25 & 0.25 & 0.25 & 0 & 0\\ 0.25 & 0 & 0.25 & 0.25 & 0 & 0.25\\ 0 & 0 & 0 & 0 & 0.5 & 0.5\\ 0 & 0 & 0 & 0 & 0.5 & 0.5 \end{pmatrix}$ with states $S=\left\{0,1,2,3,4,5\right\}$

Calculate the average return time to state $1$, e.g. $\mu_1=E(T_1 \mid X_0=1)$ where $T_1 = \inf\left\{n \geq 1: X_n=1\right\}$

I'm learning for an exam and here is another example task I tried. But I don't know if my thoughts and solution are correct?

So I don't really understand that weird notation at the bottom of the yellow marked text but luckily, it's mentioned that state $1$ is meant by it.

For this reason, I created following linear system:

I: $h(0) = 1+0.5h(0)+0.5h(1)\;\;\;\;$ by that I mean: If we are in state $0$, we go to state $0$ with a probability of $0.5$ and with probability of $0.5$, we go to state $1$

II: $h(1) = 0\;\;\;\;$ because that's the state where we end (in $1$)

We do not need further equations because every other value in row $1$ is zero anyway.

Solving the small system above: $h(0) = 1+0.5h(0)+0.5h(1) \Leftrightarrow h(0)=1+0.5h(0) \Leftrightarrow 0.5h(0)=1 \Leftrightarrow h(0)=2$

Now to get the average return time to state $1$, we do

$x=1+h(0) = 1+2=3$

Thus, the average return time to state $1$ is $3$.

$\endgroup$
3
$\begingroup$

That's not quite right, because there is a possibility to go from $1$ right back to itself. You should have $x=1+0.25h(0)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.