3
If I'm not mistaken, the steady-state vector satisfies $$p_n=\frac{2^n}{\prod_{k=1}^n(2^k-1)}p_0\tag1$$ The sum of the probabilities must equal $1$, and the series converges very rapidly. Summing the first few terms gives $$p_0\approx 0.20971122089755811$$
You can check this answer by substitution of $(1)$ into the formula in Ben's answer.
I arrived at ...
3
First of all, the approach you describe for finite state-space Markov chains is inefficient; the steady state can be more easily computed as the left-eigenvector of $P$ associated with the eigenvalue $1$. That is, $\mathbf p$ is the solution to the equation $(P - I)^T\mathbf p = 0$ normalized so that its entries are non-negative with sum $1$.
The analogous ...
1
This is by no means a complete answer. If anything, it may be empirical evidence that this is a hard problem.
I've been trying to get some computational results for the problem of determining the probability that some state $(a,b,c,d)$ is ever reached. If this step is reached, the first coordinate was increased $a-1$ times, the second $b-1$ times, and so ...
Only top voted, non community-wiki answers of a minimum length are eligible
