# Markov Chain? (definition and/or material recommendation needed)

I am trying to figure out whether there are a specific definition or materials I can study for analyzing the following system.

From Markov Chain definition, I think we need to have fixed probability transition matrix. However, I am interested in calculating the hitting time when we have dynamic probability transition matrix.

For example, let's imagine a simple discrete random walk with states $$i \in (-\infty, 0].$$ Hence, 0 is the absorbing state. However, $$p^t_{i,j}$$ is monotone increasing with $$t$$. Then we are sure about the convergence. When we want to know $$\mathbb{E}[T_{-1,0}]$$, which is the expected hitting time, what we can do?

I will be really appreciated for any recommendation to the materials I can study. Thank you!

On the state space $$(-\infty,0] \times [0,\infty)$$, consider a new Markov chain (knows as the space-time chain) with transition matrix $$\widetilde{P}$$ defined by $$\widetilde{P}\Bigl((i,t),(j,t+1)\Bigr)=p_{ij}^t$$ and $$\widetilde{P}\Bigl((i,t),(j,s)\Bigr)=0$$ for $$s \ne t+1$$. The space-time chain has fixed transition matrix, and you are asking for $$h(-1,0)$$, where $$h(v)$$ is the mean hitting time of the line $$\{0\} \times [0,\infty)$$ from the initial state $$v$$. Now the setting is more standard, and methods you know to find hitting times apply. For instance, the $$h(v)$$ satisy linear equations.