Imagine I have some Markov process with stationary distribution $\pi$ and a mixing time of $\tau$ after which $|Prob[x=s_i] - \pi(s_i)| \leq \epsilon$.
Can I assume the following:
A state $(x=s_i)$ will be sampled with probability at least $\pi(s_i)$ in time $\Theta(\tau)$? Note that I'm talking about the average case, and saying nothing about variance.
If this assumption is false, how do we rectify this with the notion that we converge on $\pi$ after some appropriate amount of mixing time $\tau$?
