1
$\begingroup$

I have given a Markov Chain $X_n$ with the state space $\{0,1,2\}$ and the transition Matrix $$P= \begin{Bmatrix} 0.3 & 0.2 & 0.5 \\ 0.5 & 0 & 0.5 \\ 0.2 & 0.1 & 0.7 \end{Bmatrix} $$

Given a function $f(x)=x^3$ for $x\in\{ 0,1,2 \}$. Is $f(X_n)$ a Markov Chain?


I don't know what to do here. I looked in my lectures but can't find anything on what a function does to a Markov Chain. I know that the state space changes to $\{0,1,8\}$ but what does the function do with the transition probabilities? How does the transition matrix change? Is $f(X_n)$ even a Markov chain? And if not how do you see this?

$\endgroup$
  • 2
    $\begingroup$ Hint: The map $x\mapsto x^3$ on $\{0,1,2\}$ is injective. $\endgroup$ – Math1000 Feb 11 '16 at 20:34
  • $\begingroup$ So what does that mean? I still don't know how the transition matrix changes. $\endgroup$ – Lawg Feb 11 '16 at 20:47
  • $\begingroup$ It doesn't change the transition probabilities. See @grand_chat's answer. $\endgroup$ – Math1000 Feb 12 '16 at 1:31
2
$\begingroup$

Since $f$ is one-to-one, the new sequence $f(X_n)$ is still a Markov chain. To prove this, argue that the transition matrix is the same as before, only the rows and columns are labelled with the new state space $f(0), f(1), f(2)$ instead of $0, 1, 2$.

$\endgroup$
  • $\begingroup$ ok I think I understand why an one-on-one function of the Markov chain is still a markov chain. But why does the transition matrix stay the same? The problem is that I can't see a connection between $f(X_n)$ and the transition probabilities.... $\endgroup$ – Lawg Feb 11 '16 at 21:15
  • 1
    $\begingroup$ For example: $P(f(X_2)=f(x_2) \mid f(X_1)=f(x_1)) = P(X_2=x_2\mid X_1=x_1)$ for any $x_1$ and $x_2$. The reason is $f$ is one-to-one, so the event $f(X_2)=f(x_2)$ is the same as the event $X_2=x_2$, and so on. $\endgroup$ – grand_chat Feb 11 '16 at 21:19
  • $\begingroup$ thanks much! One last question: if it was for example $f(x)=x^2$ with the state space $\{-1,0,1\}$ and the same transition matrix. Is then $f(X_n)$ a markov chain? The new state space is then $\{0,1\}$ and $f$ is not one-on-one. $\endgroup$ – Lawg Feb 11 '16 at 21:48
  • $\begingroup$ The answer is maybe yes, maybe no. You have to calculate the transition probabilities to find out. Call the new sequence $Y_1, Y_2, \ldots$. To be Markov the transition probs $P(Y_2=y_2\mid Y_1=y_1)$ have to be independent of your starting distribution $P(Y_1=y_1)$, for all possible values of $y_1$ and $y_2$. This calculation will be tedious. For example calculate $P(Y_2=1, Y_1=1) = P(Y_2=1, X_1=1) + P(Y_2=1, X_1=-1)$ = $P(Y_2=1\mid X_1=1)P(X_1=1) + P(Y_2=1\mid X_1=-1)P(X_1=-1)$ = $.9P(X_1=1) + .8P(X_1=-1)$. Divide this by $P(Y_1=1)=P(X_1=1) + P(X_1=-1)$ to obtain $P(Y_2=1\mid Y_1=1)$. $\endgroup$ – grand_chat Feb 11 '16 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.