0
$\begingroup$

Let $X$ be a Markov chain with state space $E = {1, 2, 3, 4}$ and transition matrix

$$P = \begin{bmatrix} 0 & 0 & 1 & 0 \\ 0 & 0.4 & 0.6 & 0\\ 0.8 & 0 & 0.2 & 0\\ 0.2 & 0.3 & 0 & 0.5\\ \end{bmatrix} $$

Compute

$$E[f(X_5)f(X_6)|X_4 = 4]$$

for the function $f$ with values $2, 4, 7, $ and $3$ at states $1, 2, 3, $ and $4$ respectively.

My solution:

I assume you can find $E[f(X_5)]E[f(X_6)]$ due to independence.

For $E[f(X_5)]$, since step begins in state $4$ then proceeds one step, I looked at the fourth row and considered all possibilities. State $4$ to state $1$ has probability $0.2$ with value $2$. Likewise, state $4$ to state $2$: $0.3$ with $4$, state $4$ to state $4$: $0.5$ with $3$. Then taking the product of each pair then summing all you get $3.1$.

Then, for $E[f(X_6)]$, you begin in state $4$ but need to take two steps. You can square the transition matrix. You need only consider the fourth row since you know you begin in state $4$. Thus, take row $4$ and multiply by each column, obtaining:

$$P = \begin{bmatrix} 0.1 & 0.27 & 0.38 & 0.25\\ \end{bmatrix} $$

This part seems correct, as the elements sum to $1$.

Then as before take each probability with its corresponding function value in each state, find the products and sum, and I obtained $4.69$.

Finally take the product $(3.1)(4.69) = 14.539$.

The answer should be $14.41$. Where are my errors? And how to obtain the correct solution?

$\endgroup$
  • $\begingroup$ It is not true that $X_n$'s are independent . $\endgroup$ – Kavi Rama Murthy Oct 2 '19 at 23:22
1
$\begingroup$

Your independence assumption is not valid.

Hints for a correct approach: $P(X_6=i,X_5=j|X_4=4)=p_{4j}p_{ji}$ from Markov property. Compute this for all possible values of $i$ and $j$ and the use the equation

$E(f(X_6)f(X_5)|X_4=4)=\sum_{i,j} f(i)f(j) P(X_6=i,X_5=j|X_4=4)$.

$\endgroup$
  • $\begingroup$ But why would you go from step four to five to six in your probability? It seems like step five and step six are treated as two separate functions in the expectation. $\endgroup$ – Vahan Oct 3 '19 at 0:36
  • $\begingroup$ Thank you I arrived at the answer using that approach. $\endgroup$ – Vahan Oct 3 '19 at 5:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.