I have just started to study Markov Chains from the last chapter (Page-$130$) of the book of Seymur Lipchuz.

I haven't understood this particular sentence: "We now consider a sequence of trials whose outcomes, say, $X_1$,$X_2$, ..." .

What do those random variables represent, say, in the following matrix?

A Markov Chain $(X_n)_n$ has the following transition matrix:
$$P = \begin{bmatrix} 0.1 & 0.3 & 0.6\\ 0 & 0.4 & 0.6\\ 0.3&0.2&0.5 \end{bmatrix}$$ with initial distribution $\alpha = (0.2, 0.3, 0.5)$.

What do the terms $X_n$ mean in this context?

What does the term, say, $P(X_9 = 2|X_1 = 2, X_5 = 1, X_7 = 3)$ mean in this context?

what does the term $EX_2$ mean in this context?

  • 1
    $\begingroup$ As it says on page 130, "We now consider a sequence of trials whose outcomes, say, X1,X2, ". The $P$ expression is a conditional probability. I don't see $EX_2$ anywhere on that page but I would guess it means the expected value of $X_2$. $\endgroup$ – John Douma Mar 10 at 5:28
  • $\begingroup$ I recommend you read the examples immediately following the definitions. It may be a good idea to use multiple sources if you are learning this on your own. $\endgroup$ – John Douma Mar 10 at 5:49

$X_i$ is the result of the $i$th "trial" in a sequence. The Markov Chain has three states and the transition matrix $P$ gives the probabilities of moving between states for any particular step in the chain (from state row# to state column#). Note that the row sums of $P$ all equal $1$.

For example, if the state at time $n$ is $1$, the probabilities for the state at time $n+1$ are given by the first row. The probability of state $n+1$ equalling one, given that the state $n$ was one, equals 0.1. This is written $P(X_{n+1} = 1|X_n=1) = 0.1$. The rest of the row gives the other possibilities for transition from state one: P(state $1 \rightarrow$ state $2$) $= 0.3$, P(state $1 \rightarrow$ state $3$) $=0.6$.

Lipchuz's point (ii) is important: The outcome of any trial depends at most upon the outcome of the immediately preceding trial.

This means that the conditional probability: $$𝑃(𝑋_9=2|𝑋_1=2,𝑋_5=1,𝑋_7=3)$$ is supplying unnecessary information. It might be translated to: What is the probability of moving from "state 3" to "state 2" in exactly $9-7=2$ steps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.