Markov chain with infinitely many states I understand that a Markov chain involves a system which can be in one of a finite number of discrete states, with a probability of going from each state to another, and for emitting a signal.
Thus,an $N \times N$ transition matrix and an $N \times N$ emission matrix of real numbers adequately describe a Markov chain with $N$ states and $M$ emissions.
Is it possible to have a Markov chain with an infinite number of states? For example, if $N=2$ is a LED that can glow blue or red, $N=\infty$ would be a LED which can glow a color that is any mixture of blue or red.
Can't an infinitely-large matrix be represented by a function of two variables (the two indices)?
 A: Yes, Markov processes with infinitely many states are indeed considered.   Random walks are a common example.  The term "Markov chain" is often reserved for the case of a discrete state space.  If the state space is finite, it's a "finite Markov chain".  See e.g. http://www.statslab.cam.ac.uk/~james/Markov/
A: If the state space $S$ is countably infinite, finite sums are replaced by sums of convergent series and square matrices of finite size by arrays indexed by $S\times S$, but many formulas are unchanged, at least formally.
By contrast, here is a famous example of a Markov chain on an uncountable state space, namely, on $S=[0,1]$. Let $(U_n)_{n\geqslant1}$ and $(V_n)_{n\geqslant1}$ denote two independent i.i.d. sequences with $U_n$ uniform on $[0,1]$ and $V_n$ uniform on $\{0,1\}$, and let $X_0$ denote any random variable with values in $[0,1]$, independent of  $(U_n)_{n\geqslant1}$ and $(V_n)_{n\geqslant1}$. 
One defines $(X_n)_{n\geqslant0}$ recursively, for every $n\geqslant0$, by 
$$X_{n+1}=U_nX_n+(1-U_n)V_n$$
Then $(X_n)_{n\geqslant0}$ is a Markov chain on $[0,1]$, whose stationary distribution can be computed explicitly. 
In words, $V_n$ describes the decision to choose the interval $I_n=[0,X_n]$ if $V_n=0$ or $I_n=[X_n,1]$ if $V_n=1$, then $X_{n+1}$ is uniformly distributed in this (random) interval $I_n$.
