The task

Let $\xi_i, i \in \mathbb{Z}_+$ be i.i.d. random variables on $\mathbb{R}$ with a probability density function $\rho(x) > 0$. Denote by $\eta_n$ the minimum of r.v.s $\xi_i$ for $i \leq n$. Is $\eta_n$ a discrete time Markov process? Find transition probabilities if it is the case.

What I can do

I can prove that $\eta_n$ is a Markov chain in case of $\xi_i$ are discrete-valued r.v.s. In fact it is quite easy to check that \begin{equation} \mathbb{P}(\eta_{n+1} = x \mid \eta_0 = x_0, \dots, \eta_n = x_n) = \mathbb{P}(\eta_{n+1} = x \mid \eta_n = x_n), \end{equation} since $\eta_{n+1} = \min(\xi_1, \dots, \xi_{n+1}) = \min(\eta_n, \xi_{n+1})$.

For the continuous-state case I can find a transition probability function (or a stochastic kernel) $\kappa(x_0, A)$: decompose $A$ as a disjoint union of three subsets $A_<, A_=$ and $A_>$ which are subsets with elements less, equal and greater than $x_0$, respectively. If $A = A_>$ then $\kappa(x_0, A) = 0$, since process can not grow. Otherwise if $A_=$ is empty then $\kappa(x_0, A) = \int_{A_<}{\rho(t) \mathrm{d}t}$ and for the last case $A_= = \{x_0\}$ we obtain $\kappa(x_0, A) = \int_A{\rho(t) \mathbb{d}t}$.

My professor says than since Markov property (the past and the future are independent if we fix the present) is nonconstructive, so one should find a transition probabilities to verify if the process is Markovian, but I don't quite understand what should I verify about $\kappa$.

Oliver Knill in his book "Probability and Stochastic Processes with Applications" gives the following definition of a discrete time Markov process: given a probability space $(\Omega, \mathcal{A}, \mathbb{P})$ with a filtration $\mathcal{A}_n$ of $\sigma$-algebras. An $\mathcal{A}_n$-adapted process $X_n$ with values in $S$ is called a discrete time Markov process if there exists a transition probability function $\kappa$ such that \begin{equation} \mathbb{P}[X_n \in B \mid \mathcal{A}_k](\omega) = \kappa^{n-k}(X_k(\omega), B), \end{equation} where $\kappa^m$ is defined as follows: \begin{equation} \kappa^1(x, B) = \kappa(x,B), \quad \kappa^{k+1}(x,B) = \int_S{\kappa^k(y, B)\kappa(x, \mathrm{d}y)}, \end{equation} where $\int_S{\kappa(x, \mathrm{d}y)}$ is integration on $S$ with respect to the measure $\kappa(x, \cdot)$.

This definition looks quite hard to verify for me (I am not even sure if it is possible to say something about the filtration $\mathcal{A}_k$), so if there any sufficient properties or theorems which will help to complete the task?

  • $\begingroup$ Since $$ \eta_{n+1} = \eta_n\cdot\mathsf 1_{\{\xi_{n+1}>\eta_n\}} + \xi_{n+1}\cdot\mathsf 1_{\{\xi_{n+1}\leqslant \eta_n\}} = g(\eta_n, \xi_{n+1}) $$ with $g$ measurable, the Markov property holds. $\endgroup$ – Math1000 May 18 '18 at 14:46
  • $\begingroup$ @Math1000, could you please elaborate a little more? I agree with this equality since I can take $g = \min$, but how does the Markov property $\mathbb{P}(\eta_t \in A \mid \mathcal{A}_s) = \mathbb{P}(\eta_t \in A \mid \eta_s)$ follows? Do you use different definition of Markov property? P.S. I'm not very good with filtrations, so I can miss something obvious $\endgroup$ – Vanzef May 18 '18 at 15:13
  • 1
    $\begingroup$ $\eta_{n+1}$ is a measurable function of $\eta_n$ and $\xi_{n+1}$. So conditioned on $\eta_n$, $\eta_{n+1}$ is independent of $(\eta_1,\ldots,\eta_{n-1})$. $\endgroup$ – Math1000 May 18 '18 at 17:23

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.