# probability terminology for parameter in a Markov process

Suppose $$P(\text{feature present at time} \ t \ \text{and} \ t+\Delta t) = \beta^{2}+\beta(1-\beta) \exp(\Delta t/\tau)$$

where $\tau = 1/(\pi_{01}+\pi_{10})$. What is $\tau$?

-
Generally speaking, it looks like a normalizing or scaling factor. – anon Aug 24 '11 at 8:09
Do you mean $\exp(-\Delta t / \tau)$, with a minus sign? Otherwise the probability could be > 1. – zyx Aug 24 '11 at 8:36
Specifically, tau measures the scale of the decay of correlations, see scholarpedia.org/article/Decay_of_correlations – Did Aug 24 '11 at 8:40

I don't recognize the formula exactly but $1/\tau$ is proportional to a rate at which something happens and $\tau$ a time scale (as Didier suggests in the comment).

Imagine a light or other two-state system (e.g., feature on / feature off) that is being activated, de-activated, or reversed, at random times according to a random process with some characteristic rate of occurrence of each action per unit of time. The events ON (1), OFF (0), ON-reversal (10), and OFF-reversal (01) could have rates (or probabilities) $r_0, r_1, \pi_{10}, \pi_{01}$, and the events could happen according to a Poisson process or suitable Markov chain.

If, on average, the light is on for a fraction $\beta$ of the time, then for large $\Delta t$ what happens at the two times is independent and the probability is close to $\beta^2$. For small $\Delta t$ the probability of a change of state is also small, and the probability of light being on at both times is almost the same as that of the light being on at time $t$, which is $\beta$.

$\exp(-\Delta t/\tau) = \exp (-\Delta t (\pi_{01} + \pi_{10}))$ looks like a probability that no reversals occur. For example, it is that probability for an interval of length $\Delta t$ if, on average, one person per unit time randomly walks by the light switch and a fraction $\pi_{10}$ of people always will turn the light off (if on) and $\pi_{01}$ will always turn it on (if off).

-
Your first paragraph is misleading since time scale means that t and tau appear only as t/tau and since there is more than that in the scale of the decay of correlations that I invoked. Namely, the fact that P(feature at t and t+s) is roughly P(feature at t) if s is small when compared to tau, and is roughly the product P(feature at t).P(feature at t+s) if s is large when compared to tau. Hence, the fact that feature at t and feature at t+s become roughly decorrelated when s crosses the time scale tau and not before. – Did Aug 24 '11 at 9:46
Here tau is a measure of the decorrelation time, and it does appear in the formulas as a time scale -- once as a ratio of times t/tau, and once as a waiting time of the form 1/(probability). A unit-less tau not related to time is a correlation decay parameter rather than a "scale" of anything. I agree that in the third paragraph, "small" and "large" dt properly refer to small and large values of dt/tau, and that talking only about the limiting behavior as $\Delta t \to 0$ and $+\infty$ obscures this nuance. Purpose of that paragraph was not tau but reconstructing the stochastic model. – zyx Aug 24 '11 at 10:49
Nobody talked about a unitless tau. The trouble is that you refer to my comment a propos something (unit of tau = unit of t) which is (1) trivial and (2) not what my comment says. Never mind. – Did Aug 24 '11 at 11:15
@Didier: you seem to agree that tau has units of time (whether or not this was related to your intended point), and I agree that "small" and "large" values of $\Delta t$ (for finite time changes, not the limit of zero or infinite dt) properly refer to $\Delta t / \tau$ rather than $\Delta t$ alone. I also agree that the concept of "correlation decay scale" is valuable here but for the present discussion it boils down to $t/\tau$ vs $t$ – zyx Aug 24 '11 at 19:14
You persist mentioning irrelevant points nobody talks about, and evading what my comment says. Please cancel the mention of my name in your post. – Did Aug 30 '11 at 11:25

As suggested earlier, this formula can be realized by a continuous-time Markov process with two states.

Start with a discrete-time Markov chain on states 0 and 1, with the matrix of transition probabilities $M = \pi_{ij}$. Construct the continuous-time Markov chain that makes a transition according to $M$ at Poisson distributed times, with an average rate of one transition per unit of time. The continuous process has infinitesimal generator $M - I$, ie. the probability distribution evolves by $e^{t(M-I)}$.

$M$ has eigenvalues $1$ and $(1-(\pi_{01}+\pi_{10}))$ and its spectral gap (also called eigenvalue gap) is the difference between these, $\pi_{01}+\pi_{10}$. The reciprocal of this gap, written as $\tau$ in the formula, is known as the relaxation time in the Markov chain literature (at least for reversible chains, and sometimes for non-reversible chains such as our $M$).

If the $\pi_{ij}$ are chosen so that $\beta = \pi_{01} / (\pi_{01} + \pi_{10})$ then

• the stationary distribution of the chain assigns probability $\beta$ to state $1$;

• given that the chain is in state $1$ at time $t$, the probability of being in state $1$ at time $t + |\Delta t|$ is $\beta + (1-\beta)\exp(-|\Delta t|(\pi_{01}+\pi_{10}))$.

This is equivalent to the formula in the question after correcting the signs in the exponent. The Markov process should be started in its equilibrium distribution, which means either starting the chain at time $-\infty$, or starting at finite time $t_0$ after initializing the chain to state 1 or 0 with probabilities $\beta$ and $1 - \beta$ respectively.

-