Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Give an example of a stochastic process $X_{n}$ that is not a Markov chain, such that $P_{y}(N(y)=\infty)=0$ but $E_{y}N(y)=\infty$

share|improve this question
    
What's $N(\bullet)$? –  Inquest Feb 8 '13 at 6:10
    
Number of visits to a state? This, and much much more, is lacking from the question. For example, what did you try, which non Markov processes do you know, ... –  Did Feb 8 '13 at 6:47

1 Answer 1

Here is an example related to your question.

Consider some i.i.d. positive integer valued $(Y_x)_{x\geqslant1}$. Let $Z_0=0$ and $Z_x=Y_1+\cdots+Y_x$ for every $x\geqslant1$. For every $n\geqslant0$, let $X_n=\max\{x\geqslant0\mid Z_x\leqslant n+Z_{X_0}\}$.

In words, $X_{n+1}$ is almost surely either $X_n$ or $X_n+1$, and the process $(X_n)_{n\geqslant0}$ stays at $x$ during $Y_x$ time steps.

The following holds:

  • The number of visits to $x\geqslant X_0$ is $Z_x$.
  • The number of visits to $x\geqslant X_0$ is almost surely finite.
  • The number of visits to $x\geqslant X_0$ is integrable iff $Z_x$ is integrable.
  • The process $(X_n)_{n\geqslant0}$ is a Markov chain iff, for every $x\geqslant X_0$, $Z_x$ has a geometric distribution.
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.