Understanding hitting probabilities of Markov chains My lecture notes says that "the hitting probability of a state $y$ starting from a state $x$ is defined as $$\rho_{xy} = \mathbb{P}_x(T_y < \infty),$$ where $T_A$ is the hitting time of the set $A \subseteq S$ and is defined as $$T_A = \inf\{t > 0 : X_t \in A\}.$$ In other words, $\rho_{xy}$ is the probability that, starting from $x$, we will eventually be in $y$ at some future time."
To understand this more deeply, I asked my professor if we could say that $$\rho_{xy} = P(x, y) + P^2(x, y) + P^3(x, y) + ...,$$ where $P^n(x, y)$ is the $n$-step transition probability from $x$ to $y$. However, he said that this is not quite right because we would be over-including cases since, for example, $P^2(x, y)$ would include $P(x, y)$.
This makes sense, so I asked if we could instead say that $$\rho_{xy} = \lim_{n \to \infty} P^n(x, y).$$ However, he then said that this only holds for certain cases (which we will apparently discuss soon).
Thus, my train of thought has now left me wondering why $$\rho_{xy} = \lim_{n \to \infty} P^n(x, y)$$ does not hold in general.
I am only taking an introductory module in stochastic processes, so any intuitive explanations will be greatly appreciated :)
 A: One thing I have to mention which could be helpful for your question.
$\rho_{xy}=\lim\limits_{n\to\infty}P^n(x,y)$holds when state y is an absorbing state.
It's because we have $\forall n\ge 1, P^{n}(x,y)=\sum\limits_{m=1}^{n}\mathbb{P}_x(T_y=m)P^{n-m}(y,y)$(it's obvious).
When y is an absorbing state, $P^{n}(y,y)=1 , \forall n$.
So $P^{n-m}(y,y)=1$ and $P^{n}(x,y)=\sum\limits_{m=1}^{n}\mathbb{P}_x(T_y=m)$ hold.
Let n goes to zero,we have $\rho_{xy}=\mathbb{P}_x(T_y<\infty)=\sum\limits_{m=1}^{\infty}\mathbb{P}_x(T_y=m)=\lim\limits_{n\to\infty}\sum\limits_{m=1}^{n}\mathbb{P}_x(T_y=m)=\lim\limits_{n\to\infty}P^{n}(x,y)$.
I guess that is what your professor whats to convey.
A: In the direction of what you first tried, letting $B$ denote the set of all states except $y$, you have
$$
\Bbb P_x(T_y<\infty) = P(x,y)+\sum_{x_1\in B}P(x,x_1)P(x_1,y)+\sum_{x_1.x_2\in B}P(x,x_1)P(x_1,x_2)P(x_2,y)+\cdots.
$$
More compactly, writing $Q(x,z)=P(x,z)1_B(z)$, you have
$$
\Bbb P_x(T_y<\infty)=\sum_{n=0}^\infty [Q^nP](x,y).
$$
