# Markov chain exit time

Consider a reversible markov chain $X_t$ defined on a square lattice, with transition probabilities defined between adjacent vertices. Take a square subset of the lattice and call it $V$. Let $dV$ be the set of vertices attached by incoming edges into $V$, ie the hair around the square. Let $\tau$ be the first time that X hits a vertex in $dV$. Suppose I start $X$ at a point $x$ in $dV$ and condition it to enter $V$ on the next step. What can one say about the joint probability that $\tau=n$ and $X_\tau=y$? In particular, are there good references for this somewhere? I believe the proper term for this is "first return time" yet a google search gives ambiguous results. Thanks!!

-

To start at $x$ in $dV$ and to condition to enter $V$ on the next step is equivalent to assuming that one starts from the unique vertex in $V$ which is a neighbor of $x$. Hence from now on I will assume that the Markov chain starts from a vertex in $V$. Of course, the case which interests you is when this vertex has a neighbor in $dV$ but I will not need this hypothesis. Likewise the structure of the graph has no relevance so I consider a general Markov chain, with transition probability kernel $p(\cdot,\cdot)$.
For every $n\ge1$, $x$ in $V$ and $y$ in $dV$, call $h_n(x,y)$ the probability that $X_\tau=y$ and $\tau=n$ when the Markov chain starts from $x$. Then, $h_1(x,y)=p(x,y)$ and the Markov property of the Markov chain at time $1$ yields that, for every $n\ge1$, $$h_{n+1}(x,y)=\sum_{z\in V}p(x,z)h_n(z,y).$$ One can also encode the whole sequence $(h_n(x,y))_n$ through a single function $H_{x,y}$, as $$H_{x,y}(s)=\sum_{n\ge1}h_n(x,y)s^n.$$ Then, the recursions given above are equivalent to the relations $$H_{x,y}(s)=sp(x,y)+s\sum_{z\in V}p(x,z)H_{z,y}(s).$$ Finally, for every $n\ge1$, $$h_n(x,y)=\sum_cp(c),$$ where the sum over $c$ enumerates the paths $c$ of length $n$ which start from $x$ and stay in $V$ until their endpoint $y$, and $p(c)$ is the product of the transition probabilities from one vertex of $c$ to the next one.