# Markov chain with known duration

Let's say there are 2 states (A & B), where the probability of going from state A to B in interval i is $$P_{ab}$$. State B always lasts for n intervals, then always goes to back to state A.

I want to know what is the probability of being in state B at any given interval. Is it correct to calculate that probability like this:

$$P(B)= (P_{ab}*n)/(1+P_{ab}*n)$$ ?

I see Markov chains being applied to problems similar to this, but I just can't seem to find examples of this exact type of problem.

I don't have enough reputation to embed a picture, but the link shows a diagram of the situation I'm describing.

https://i.stack.imgur.com/Eplf6.png

New contributor
Tage Wærdahl is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

One option is to make a larger Markov chain that embeds the history information in the state space. Rather than having 1 State that corresponds to B, you would have $$n$$ states that correspond to how many intervals you've been in State B (aka "dummy states" that each record how long you've been in state B). Whenever you're in state B, each time interval you would transition to the next dummy state B up until the n-th one. Then transition back to A.

I.e., you would have the following states:

• State $$A$$

• State $$B_1$$

• ...

• State $$B_n$$

Transition probabilities:

• State $$A$$ to State $$B_1$$ $$:P_{ab}$$

• State $$B_j$$ to $$B_{j+1}$$ for all $$\{j\in 1..n-1\}$$ $$:1$$

• State $$B_n$$ to $$A:1$$

This should get you your Markov chain. From here, I'll let you think about how to calculate the probability of being in State A and any of the dummy State B's.

• Good news - I worked through the math of my solution, and the answer is equivalent to the formula you gave. Apologies for initially indicating your response was wrong. Generally, it's a bad idea to consider durations within a Markov chain, but in this case, it worked out fine. – E. Tucker 2 days ago
• Thank you for the response Mr./Mrs Tucker, That is wonderful :D Got that equation from a completely different approach, but I wasn't 100% confident in it being right.. If its not too much to ask, could I see the work to get there? A bit rusty I must admit, spent some hours trying to relearn steady state Markov chains, but didn't quite get there myself :p I intend to use this "equation" in my masters, so thank you so much for your thourough response. – Tage Wærdahl 2 days ago
• After a good rest I got to the formula myself. Again, thank you for explaining the procedure. I would like to acknowledge your help in the acknowledgement section of my report. Please hit me back, to let me know if that is okay, and how you would like to be titled. – Tage Wærdahl yesterday
• Glad you could get it! I hadn't had the time to type it up. That's very kind, you can refer to me as: Emily Tucker, MSE. Best wishes with your masters! – E. Tucker yesterday