Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Give an example of a stochastic process $X_{n}$ that is not a Markov chain, such that $P_{y}(N(y)=\infty)=0$ but $E_{y}N(y)=\infty$

share|cite|improve this question
What's $N(\bullet)$? – Inquest Feb 8 '13 at 6:10
Number of visits to a state? This, and much much more, is lacking from the question. For example, what did you try, which non Markov processes do you know, ... – Did Feb 8 '13 at 6:47

Here is an example related to your question.

Consider some i.i.d. positive integer valued $(Y_x)_{x\geqslant1}$. Let $Z_0=0$ and $Z_x=Y_1+\cdots+Y_x$ for every $x\geqslant1$. For every $n\geqslant0$, let $X_n=\max\{x\geqslant0\mid Z_x\leqslant n+Z_{X_0}\}$.

In words, $X_{n+1}$ is almost surely either $X_n$ or $X_n+1$, and the process $(X_n)_{n\geqslant0}$ stays at $x$ during $Y_x$ time steps.

The following holds:

  • The number of visits to $x\geqslant X_0$ is $Z_x$.
  • The number of visits to $x\geqslant X_0$ is almost surely finite.
  • The number of visits to $x\geqslant X_0$ is integrable iff $Z_x$ is integrable.
  • The process $(X_n)_{n\geqslant0}$ is a Markov chain iff, for every $x\geqslant X_0$, $Z_x$ has a geometric distribution.
share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.