After answering the question Expectation of a stopping time uniquely determined by a function I was looking for the literature on the mean hitting/exit time for a discrete-time Markov process. In Meyn and Tweedie, Durett and some other books I haven't find such an information. Google also didn't help much for the discrete-time setting.
I guess that for the uncountable state space, mean hitting time equation shares the same properties as for the countable state space - but I would prefer to read a book chapter on this topic.
Dynkin briefly discussed it, though for the continuous time and too briefly.