Expected value for Markov Chain Let $(X_n)_{n\geq0}$ is a Markov chain with state space $I=\{0,1,2,3,4,5,6 \}$ and transission matrix with transition probabilities $P_{i,i+1}=0.5, P_{i,0}=0.5$ and $P_{6,6}=1$ where $I\in\{0,1,2,3,4,5\}$. If $T=inf\{m:X_m=6\}$
$$P=\begin{bmatrix} 0.5 & 0.5 & 0 &0 &0&0&0 \\  0.5 & 0 & 0.5 & 0&0&0&0\\ 0.5 & 0 & 0&0.5&0&0&0 \\0.5&0&0&0&0.5&0&0 \\0.5&0&0&0&0&0.5&0\\0.5&0&0&0&0&0&0.5\\0&0&0&0&0&0&1    \end{bmatrix}$$
How can I compute $E_3(T)$
 A: What you ask for can be computed by a standard result in Markov chain theory. In your case, you ask for the expected hitting time of $6$ given that you start at state $3$. Instead, compute $k_n:=E_n(T)$, i.e. the expected hitting time of $6$ starting from $n=0,1,...,6$. The theory then tells us, for instance in Markov Chains by J.R. Norris section 1.3, that $k_6=0$ and for $n\neq 6$, that
$$k_n=1+0.5(k_{n+1}+k_0).$$
Solving this recurrence relation will give you $k_n$ for each $n$ and then one can find $k_3$. Note, this is the exact relation that user Omnomnomnom suggested to investigate.
In general, if $(X_n)$ is a discrete time Markov chain on state space $S$ with probability transition matrix $P$, and $T_A:=\inf \{n : X_n\in A\}$, where $A\subset S$, then $k_i:=E_i(T_A)$ satisfies,
$$k_i=0, \,\text{ if } i\in A$$
and
$$k_i=1+\sum_{j\notin A} p_{ij} k_j, \, \text{ if } i\notin A.$$
By the way, the notation $E_3(T)=E_3(X_n=6)$ makes no sense. The expression "$X_n=6$" is not a random variable, (it is a value the RV may take), so you cannot take the expectation of it. For methods on solving the linear recurrence relations that result from the expected hitting time equation, see wikipedia , or ask a new question.
