# Question about integral notation in a Markov process + how to evaluate said integral

I'm reading Chapter 11 of Puterman's book on Markov Decision Processes (in particular, about continuous-time Markov processes). There's a lot of notation involved, but I've tried to distill the question. Puterman defines a function $$Q(t,j|s,a)$$, which, as a simple example, might equal $$Q(t,j|s,a)=\frac{1}{4}(1-e^{-\mu{t}})$$ for some $$\mu>0$$. The function $$Q$$ is a joint probability distribution in $$t\geq0$$ and $$j\in{S}$$ for finite $$S$$ (in the example above, the product of the CDF of an exponential random variable with a constant). He then writes down the integral $$\int_0^\infty e^{-\alpha{t}}Q(dt,j|s,a),$$ and asserts that the value of this integral is $$<1$$. Puterman states "[w]e use $$Q(dt,j|s,a)$$ to represent a time-differential", but I don't know what this means in the context of integration.

Question 1 What kind of integral is this? Seems like Riemann-Stieltjes or Lebesgue, but I can't tell. I thought it might be strange notation for $$\int_0^\infty e^{-\alpha{t}}Q(t,j|s,a)dt,$$ but it seems that's not the case (as then the integral can easily be $$\geq1$$).

Question 2 How do you evaluate such an integral? Is there e.g. a closed-form for the $$Q$$ defined above?

• The integral would be $$\int_0^\infty e^{-\alpha t}\frac14 \mu e^{-\mu t}\ \mathsf dt = \frac\mu{4(\alpha+\mu)}.$$ – Math1000 Jan 14 at 23:07
• @Math1000 Weird notation. Post as an answer (maybe with a small explanation?) and I’ll accept! – David M. Jan 14 at 23:18