I have seen discrete-time discrete-state Markov processes (such as random walks), continuous-time discrete-state Markov processes (such as Poisson processes), and continuous-time continuous-state Markov processes (such as Brownian motions).
I was wondering if discrete-time continuous-state Markov processes have been studied as often as the above three?
What are some of its examples then?
Discrete-time continuous state Markov processes are widely used. Autoregressive processes are a very important example.
Actually, if you relax the Markov property and look at discrete-time continuous state stochastic processes in general, then this is the topic of study of a huge part of Time series analysis and signal processing.
The most famous examples are ARMA processes, the Conditionally Heteroscedastic models, a large subclass of Hidden Markov models....
Some inventory models which assume the products are continuous are actually continuous state markov process. In operations research, people seems to care more about the structural results instead of the computation issue when they assume the state is continuous. This maybe the reason that it is not well known. Some reference of Continuous state Markov chain are: (1) S P Meyn and R L Tweedie. Markov Chains and Stochastic Stability. Cambridge University Press, 2009 (2) QUANTITATIVE ECONOMICS with Python
The references I provide may not contain many examples you are after, but they do treat theory that may help you understand examples you find elsewhere.
The book titled Probability by A.N. Shiryaev, with ISBN 978-1-4757-2541-4 (e-book ISBN 978-1-4757-2539-1) has a chapter called CHAPTER VIII; Sequences of Random Variables that Form Markov Chains.
This chapter has strong foundations in measure theoretic probability theory. That subject is dealt with in earlier chapters of the book.
It should be noted that (unless I overlooked any) the examples in this chapter deal only with discrete state Markov processes.
Note
I referred to the second edition of this book. There is a newer version of this book with ISBN 978-0-387-72206-1, which is the third edition. I refer to the second edition because I cannot find a table of contents (etc) for the third edition.
Another reference
The book The Theory of Stochastic Processes I, isbn: 978-3-642-61943-4, which can be found here has a chapter called Random Sequences.
According to the book Probability this chapter deals with Markov Processes
The existence of regular transition probabilities such that the Kolmogorov-Chapman equation is satisfied for all $x \in \mathbb{R}$ is proved in [The Theory of Stochastic Processes] Volume I, Chapter II, §4).
The title of the chapter implies that it deals with discrete time processes and a look at the preview on springer reveals that the state (phase) space is taken to be general.
An easy example is as follows: let $(X_n)$ be an iid sequence of standard Gaussian random variables (although you don't usually think of it as a discrete-time / continuous-state Markov process).
I know that this is an old question, but I just stumbled upon it and noticed that nobody has mentioned yet what I think of as maybe the most natural and simple (non-trivial) examples of discrete-time Markov processes with non-discrete state space.
Starting with a continuous time Markov process $X=(X_t)_{t\ge0}$ on any state space $E$, one can consider
the $E$-valued $\Delta$-grid chain $(X_{n\Delta})_{n\in\Bbb N}$ where $\Delta>0$,
the $E^{[0,\Delta)}$-valued $\Delta$-path-segment chain $((X_t)_{t\in[(n-1)\Delta,n\Delta)})_{n\in\Bbb N}$ where $\Delta>0$,
When looking for certain asymptotic properties of $X$, it can be sufficient (and yet much easier) to study one of these "embedded" discrete time processes. Such connections can be used even in the case where $X$ is not homogeneous but only $\tau$-periodic with respect to time, i.e. $$ \mathcal L(X_{t+k\tau}|X_{s+k\tau})= \mathcal L(X_{t}|X_{s}) \quad\text{for all $t\ge s\ge0$ and $k\in\Bbb N$.}$$ In this situation, the advantage of the grid chain or the path-segment chain is even more obvious, since they become homogeneous when one takes $\Delta=\tau$.
To illustrate this a little more, think of an SDE $$ dX_t=b(t,X_t)dt+\sigma(t,X_t)dW_t,$$ where drift and volatility are $\tau$-periodic in time. Many interesting properties of this continuous-time inhomogeneous Markov process can be derived from properties of the corresponding $\tau$-grid chain, which is a discrete-time homogeneous Markov process - and as such it is of a much simpler general nature and allows using the corresponding techniques.
