# How to verify if process is Markov stochastic process with discrete time?

Let $\xi_i, i \in \mathbb{Z}_+$ be i.i.d. random variables on $\mathbb{R}$ with a probability density function $\rho(x) > 0$. Denote by $\eta_n$ the minimum of r.v.s $\xi_i$ for $i \leq n$. Is $\eta_n$ a discrete time Markov process? Find transition probabilities if it is the case.

## What I can do

I can prove that $\eta_n$ is a Markov chain in case of $\xi_i$ are discrete-valued r.v.s. In fact it is quite easy to check that $$\mathbb{P}(\eta_{n+1} = x \mid \eta_0 = x_0, \dots, \eta_n = x_n) = \mathbb{P}(\eta_{n+1} = x \mid \eta_n = x_n),$$ since $\eta_{n+1} = \min(\xi_1, \dots, \xi_{n+1}) = \min(\eta_n, \xi_{n+1})$.

For the continuous-state case I can find a transition probability function (or a stochastic kernel) $\kappa(x_0, A)$: decompose $A$ as a disjoint union of three subsets $A_<, A_=$ and $A_>$ which are subsets with elements less, equal and greater than $x_0$, respectively. If $A = A_>$ then $\kappa(x_0, A) = 0$, since process can not grow. Otherwise if $A_=$ is empty then $\kappa(x_0, A) = \int_{A_<}{\rho(t) \mathrm{d}t}$ and for the last case $A_= = \{x_0\}$ we obtain $\kappa(x_0, A) = \int_A{\rho(t) \mathbb{d}t}$.

My professor says than since Markov property (the past and the future are independent if we fix the present) is nonconstructive, so one should find a transition probabilities to verify if the process is Markovian, but I don't quite understand what should I verify about $\kappa$.

Oliver Knill in his book "Probability and Stochastic Processes with Applications" gives the following definition of a discrete time Markov process: given a probability space $(\Omega, \mathcal{A}, \mathbb{P})$ with a filtration $\mathcal{A}_n$ of $\sigma$-algebras. An $\mathcal{A}_n$-adapted process $X_n$ with values in $S$ is called a discrete time Markov process if there exists a transition probability function $\kappa$ such that $$\mathbb{P}[X_n \in B \mid \mathcal{A}_k](\omega) = \kappa^{n-k}(X_k(\omega), B),$$ where $\kappa^m$ is defined as follows: $$\kappa^1(x, B) = \kappa(x,B), \quad \kappa^{k+1}(x,B) = \int_S{\kappa^k(y, B)\kappa(x, \mathrm{d}y)},$$ where $\int_S{\kappa(x, \mathrm{d}y)}$ is integration on $S$ with respect to the measure $\kappa(x, \cdot)$.

This definition looks quite hard to verify for me (I am not even sure if it is possible to say something about the filtration $\mathcal{A}_k$), so if there any sufficient properties or theorems which will help to complete the task?

• Since $$\eta_{n+1} = \eta_n\cdot\mathsf 1_{\{\xi_{n+1}>\eta_n\}} + \xi_{n+1}\cdot\mathsf 1_{\{\xi_{n+1}\leqslant \eta_n\}} = g(\eta_n, \xi_{n+1})$$ with $g$ measurable, the Markov property holds. – Math1000 May 18 at 14:46
• @Math1000, could you please elaborate a little more? I agree with this equality since I can take $g = \min$, but how does the Markov property $\mathbb{P}(\eta_t \in A \mid \mathcal{A}_s) = \mathbb{P}(\eta_t \in A \mid \eta_s)$ follows? Do you use different definition of Markov property? P.S. I'm not very good with filtrations, so I can miss something obvious – Vanzef May 18 at 15:13
• $\eta_{n+1}$ is a measurable function of $\eta_n$ and $\xi_{n+1}$. So conditioned on $\eta_n$, $\eta_{n+1}$ is independent of $(\eta_1,\ldots,\eta_{n-1})$. – Math1000 May 18 at 17:23