Consider the following markov chain with state space $\{1,2,3,4\}$ and transition matrix

$\begin{pmatrix}1/3&&1/3&&0&&1/3 \\ 1/4&&1/4&&1/4&&1/4 \\ 0&&0&&1/2&&1/2 \\ 0&&0&&0&&1\end{pmatrix}$

and initial distribution $\lambda=(1/2,1/2,0,0)$.

Let $T=\inf\{n\ge0:X_n \in \{1,4\}\}$.

And let $k_i:=\mathbb E_i[T]$ be the expected hitting time wrt $\mathbb P_i$.

How can I calculate the variance of $T$ wrt $\mathbb P_3$?

I know that $Var(T)=\mathbb E_{\mathbb P_3}[T^2]-(\mathbb E_{\mathbb P_3}[T])^2$

How can I determine the first summand $\mathbb E_{\mathbb P_3}[T^2]$?

  • $\begingroup$ What does "with respect to $\operatorname{P}_3$" mean? $\operatorname{P}(T = k \mid X_0 = 3) = 2^{-k} \, [k > 0]$, you don't need to know the initial distribution for that. $\endgroup$ – Maxim Jun 26 at 17:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.