I cant understand the difference between THIS and THIS.
The first one The probability of going from state i to state j in n time steps is
and gives typo and the chapman and the second one The number fii^n is the probability that we return to state i for the first time after n steps.
I cant understand the difference, especially how to calculate these two given a markov chain on paper. I guess that recurrence has something to do with "time" but what does it mean? $$ \begin{bmatrix} 1 & 0 & 0 \\ 0 & 0.7 & 0.3 \\ 0 & 0.2 & 0.8 \end{bmatrix}$$
But looking at solution i get f11(2)=0.3 * 0.2 , f11(3)= 0.3 * 0.8 * 0.2 ,f11(4)= 0.3 * 0.8 * 0.8 * 0.2 Does it selects the highest probabilities each time avoiding it self? But i cant find a formula for this. And chapman formula yields totally different result..
