# Markov Chain Expected value

Let $(X_n)_{n\in\mathbb{N}}$ be a Markov chain with State space $E=\{1,2,3\}$ and transission matrix

$$P=\begin{bmatrix} 0 & 1/3 & 2/3 \\ 1/4 & 3/4 & 0 \\ 2/5 & 0 & 3/5 \end{bmatrix}$$

How can I compute $E[X_{n+1}^3|X_n=j]$, for $j=1,2,3$.

• Looks like there's an error in your matrix as the rows should sum to $1$ Nov 24, 2014 at 18:51
• Sorry, I corrected it! Nov 24, 2014 at 18:52
• Just expand Expected value definition and as you know the condintional probability distribution function, you'll finish it : $$E[X_{n+1}^3|X_n=1]=\sum_{X_{n+1}=1}^3 X_{n+1}^3.P(X_{n+1}|X_n=1)$$ Nov 24, 2014 at 18:56

$E[X_{n+1}^3 | X_n = 1]$
$= 1^3.Pr(X_{n+1} = 1 | X_n = 1) + 2^3.Pr(X_{n+1} = 2 | X_n = 1) + 3^3.Pr(X_{n+1} = 3 | X_n = 1)$
$= 1 \cdot 0 + 8 \cdot (1/3) + 27 \cdot (2/3)$
$= ...$