UPDATE: Here is the short answer:
- The process is not necessarily Markov.
- Limiting or stationary distributions need not exist.
(Original answer below with some corrections and edits noted.)
Part I: Markov-ness. In general, this is not a Markov process. The exponential distribution is the only continuous distribution with the memory-less property. See this example. Markov processes almost always have exponentially-distributed wait times.
In order for a process to have non-exponential wait times and satisfy the Markov property, knowing the current state must give away the precise time that it entered that state. Consider the process that starts in one state, stays there for some random length of time, then jumps to another state and stays there forever. It's Markov, but not very interesting. Knowing the current state means you know whether or not the jump has occurred and can calculate the distribution of future states precisely.
If the process has two or more states and stays in each state for some non-exponential random time whence jumping to another state occurs, then knowing the current state alone will not allow you to calculate the future distribution of the process.
Part II: Stationarity. The stationary distribution does not generally depend on the expected length of time in each state as my intuition had led me to believe. I'm not completely certain, but think there could be bad enough wait-time distributions that actually cause a stationary distribution to not exist, but I'll ask that in a separate question.
EDIT: @Did has confirmed the above in comments on this answer and showed that I had an error in one of my examples. The text below is edited accordingly.
Example 1: Consider the process where $\tau_{01}$ and $\tau_{10}$ are i.i.d. Then the process has stationary distribution $\pi=[0.5, 0.5].$
If $\tau_{01}$ and $\tau_{10}$ are neither i.i.d. nor exponential, then in general, one cannot be certain of the existence of a either stationary distribution or a limiting distribution. However, as long as the wait times are non-deterministic, a limiting distribution will exist but a stationary distribution may or may not exist.
Example 2: If $\tau_{01}=a$ and $\tau_{10}=b$ with $a< b$, deterministically, then neither a limiting nor a stationary distribution exist. This is easy to see as given any starting state, the distribution will oscillate between $[0, 1]$ and $[1,0],$ and given any initial distribution $\mu$ the distribution at time $t$ will oscillate between $\mu$, $[0,1]$, and $1-\mu.$
Example 3: If either $\tau_{01}$ or $\tau_{10}$ is random (or both), then (as I expected in the question) we have limiting distribution
$$
\mu(0)=\frac{E(\tau_{01})}{E(\tau_{01})+E(\tau_{10})}, \quad
\mu(1)=\frac{E(\tau_{10})}{E(\tau_{01})+E(\tau_{10})},
$$
however if $\tau_{01}$ and $\tau_{10}$ are not identically distributed, then no stationary distribution exists.
Of course, the reader may desire rigorous proofs, but I am unable to provide them currently. I've tested several numerical examples though.
My original mistake was confusing "expected wait-time" with "average time spent in a state over a long period of time covering many jumps between states."
Part III: A 3-state, exponential example. Consider the Markov process:
$$
\begin{aligned}
0\rightarrow 1 \ \text{ at rate } \ a \\
1\rightarrow 0 \ \text{ at rate } \ b \\
0\rightarrow 2 \ \text{ at rate } \ c \\
2\rightarrow 0 \ \text{ at rate } \ d
\end{aligned}
$$
The stationary distribution is
$$
\pi=[bd, ad, bc]\cdot \frac{1}{bd+ad+bc},
$$
and $\pi(i)$ does not equal the expected length of time the process remains in state $i.$ I.e. with $\tau_i$ denoting the length of time in state $i$ waiting to jump:
$$\pi(0)\neq\frac{1}{A} E(\tau_0)=\frac{\frac{1}{a+c}}{\frac{1}{a+c}+\frac{1}{b}+\frac{1}{d}}$$
$$\pi(1)\neq \frac{1}{A} E(\tau_1)=\frac{\frac{1}{b}}{\frac{1}{a+c}+\frac{1}{b}+\frac{1}{d}}$$
$$\pi(2)\neq \frac{1}{A} E(\tau_2)=\frac{\frac{1}{d}}{\frac{1}{a+c}+\frac{1}{b}+\frac{1}{d}}$$
where $A=\sum E(\tau_i).$