Conditional probability and Markov property in the study of hitting times I'm studying Markov chains from Norris and I have some questions related to conditional probabilbity.
Let's fix some notation first.
Let $S$ be the state space (in our case $S=\mathbb{N}$ or $S$ is a finite set), let $(X_n)_{n\geq0}$ be Markov$(\lambda,P)$ (the book's notation to denote a Markov chain with initial distribution $\lambda$ and transition matrix $P$) with values in $S$. The hitting time of subset $A\subset S$, is the random variable $H^A(\omega):=\inf\{n\geq0 s.t. X_n(\omega)\in A\}$. We also set $h_i^A:=P(H^A<\infty\vert X_0=i)$ and denote $P_i(H^A<\infty):=P(H^A<\infty\vert X_0=i)$, for $i\in S$. Intuitively $h_i$ is the probability of hitting $A$ starting from $i$.
To have a concrete example, taken from Norris, let $S=\{1,2,3,4\}$ and $A=\{4\}$. We assume also $P(X_1=0\vert X_0=2)=0=P(X_1=4\vert X_0=2)$. Say we want to compute $h_2$ (omitting the dependece on $A$). Norris says that $$h_2=P(X_1=1\vert X_0=2)h_1+P(X_1=3\vert X_0=2)h_3.$$
The idea behind this formula is to make one step, which will take us in $1$ or $3$ with some probability and then to ask ourselves what's the probability that from this new position we eventually get to $4$. This makes sense intuitvely but I wanted to prove the formula just to get my hands dirty with some computation. After some attemp I gave up and I looked at Norris' proof of the general case and a wild $P_i(H^A<\infty\vert X_1=j)$ appears, $i,j\in S$.
 My questions are:  


*

*How is, $P_i(H^A<\infty\vert X_1=j)$ defined? Is it $P(H^A<\infty\vert X_1=j,X_0=i)$?

*How is the Markov property used to prove $P_i(H^A<\infty\vert X_1=j)=P_j(H^A<\infty)$?

 A: This is the "law of total probability," conditioned on living in a world where $X_0=2$.  Define $B$ as the following event: 
$$ B = \{\mbox{we visit the target set sometime on or after time 0}\} $$
So $$ \underbrace{P[B|X_0=2]}_{h_2} = \sum_{i\in S}P[B| X_1=i, X_0=2]\underbrace{P[X_1=i|X_0=2]}_{P_{2i}} $$
Then, 
\begin{align}
P[B|X_1=i, X_0=2] &= P[\mbox{vist target on or after time 0} | X_1=i, X_0=2]\\
 &\overset{(a)}{=} P[\mbox{visit target on or after time 1}|X_1=i, X_0=2]\\
&\overset{(b)}{=} P[\mbox{visit target on or after time 1}|X_1=i]\\
&\overset{(c)}{=} P[\mbox{visit target on or after time 0}|X_0=i]\\
&= h_i
\end{align}
where (a) holds because we did not
hit the target at time 0; (b) holds by the Markov property (the future is conditionally independent of the past given the current state); (c) holds  because the Markov chain is time-homogeneous. 
Putting this together gives the formula: 
$$ \boxed{h_2 = \sum_{i \in S} h_i P_{2i}} $$

In general, if $\{C_i\}_{i=1}^{\infty}$ are events that partition a sample space $S$ (so they are disjoint and satisfy $\cup_{i=1}^{\infty} C_i=S$), then for any event $B$, the law of total probability says: 
$$ P[B] = \sum_{i=1}^{\infty} P[B|C_i]P[C_i] $$
Now if we condition on living in a world where some event $F$ occurs, the conditional probability distribution given $F$ is still a valid probability distribution, and the law of total probability is the same except that everything is conditioned on $F$:
$$ P[B|F]  = \sum_{i=1}^{\infty} P[B|C_i, F] P[C_i|F] $$
