# Markov chain with infinitely many states

I understand that a Markov chain involves a system which can be in one of a finite number of discrete states, with a probability of going from each state to another, and for emitting a signal.

Thus,an $N \times N$ transition matrix and an $N \times N$ emission matrix of real numbers adequately describe a Markov chain with $N$ states and $M$ emissions.

Is it possible to have a Markov chain with an infinite number of states? For example, if $N=2$ is a LED that can glow blue or red, $N=\infty$ would be a LED which can glow a color that is any mixture of blue or red.

Can't an infinitely-large matrix be represented by a function of two variables (the two indices)?

-
Is the set of states countably infinite, or uncountable? – Rod Carvalho Sep 12 '12 at 18:35

If the state space $S$ is countably infinite, finite sums are replaced by sums of convergent series and square matrices of finite size by arrays indexed by $S\times S$, but many formulas are unchanged, at least formally.
Here is a famous example of a Markov chain on $[0,1]$. Let $(U_n)_{n\geqslant1}$ and $(V_n)_{n\geqslant1}$ denote two independent i.i.d. sequences with $U_n$ uniform on $[0,1]$ and $V_n$ uniform on $\{0,1\}$. Let $X_0$ denote any random variable with values in $[0,1]$, independent of $(U_n)_{n\geqslant1}$ and $(V_n)_{n\geqslant1}$.
Define $(X_n)_{n\geqslant0}$ recursively by $X_{n+1}=U_nX_n+(1-U_n)V_n$, for every $n\geqslant0$. Then $(X_n)_{n\geqslant0}$ is a Markov chain on $[0,1]$, whose stationary distribution can be computed explicitly. In words, $V_n$ describes the decision to choose the interval $[0,X_n]$ if $V_n=0$ or $[X_n,1]$ if $V_n=1$, then $X_{n+1}$ is uniformly distributed in this interval.