Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let $X$ be a real-valued Markov process, $$ \mathsf P_x\{X_1\in A\} = K(x,A) $$ is its transition kernel. Let $\tau = \inf\{n\geq 0:X_n\geq a\}$ be the first hitting time of the level $a$. I am interested in distribution $$ L_k(x;A) = \mathsf P_x\{X_\tau\in A|\tau = k\} $$

In the latter event we cannot just put $\mathsf P_x\{X_\tau\in A|\tau = k\} = \mathsf P_x\{X_k\in A\}$ since it doesn't hold e.g. for $A = [a,\infty)$. Any hints are appreciated.

share|cite|improve this question
up vote 1 down vote accepted

I am going to assume $K$ is nicely differentiable, so that we can talk about the probability density $dK (X_i, X_{i+1})$ associated with it.

First define $B = [a, \infty)$ and $B^c = (-\infty, a)$. Then

$ L_k(x, A) = P_x( X_{\tau} \in A | \tau = k ) = P_x ( X_{\tau} \in A \cap B | X_{\tau '} \in B^c \space \forall \tau ' < \tau )$ $ = P_x ( X_{\tau} \in A \cap B \wedge X_{\tau '} \in B^c \space \forall \tau ' < \tau ) / P_x ( X_{\tau '} \in B^c \space \forall \tau ' < \tau ) $

Each of these probabilities is expressible in terms of repeated integrals of $dK$ (over $(B^c)^{k-1}$), with the numerator having a final factor of $K(X_{k-1}, A)$ in it.

share|cite|improve this answer

I think this is solvable in continuous time with discrete state space (at least in principle) if you know the infinitesimal generator of the process (under some regularity conditions).

As this kind of question is closely related to barrier option in finance you might have a look at this article by Mijatovic and Pistorius

Maybe you can adapt their approach to your context.


share|cite|improve this answer
Double downvote ) that's interesting. – Ilya Sep 26 '11 at 14:01

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.