# Combined Markov Chain of N identical Markov Chains

I have a markov chain from https://ieeexplore.ieee.org/document/9766097 as follows The state transition matrix is as below The analysis pertains to packet drop rate for ONE device with certain arrival($$\lambda$$) and transmission success probability($$\mu_1$$) where $$\bar\lambda$$ stands for no arrivals ($$1-\lambda$$) and $$\bar\mu_1$$ pertains to transmission failure probability ($$1-\mu_1$$).

The drop rate for ONE device is given by $$\pi_d\bar\mu_1$$ where $$\pi_d$$ is the steady state probability for the markov chain with delay bound $$d$$ (after which a packet will be dropped) in state $$d$$

I need to calculate the combined drop rate for N devices which are identical and symmetrical (i.e. have the same probability for successful transmission). What is the correct approach to extend this analysis for N devices? May be it is trivial, but I am missing the point.

I think you mean expected drop rate (since actual drop rate is stochastic).

Let $$D_i$$ be the drop rate of device $$i$$. You are asking about the following quantity:

$$E\left[ \sum_{i=1}^N D_i \right]$$

Where we are assuming the $$D_i$$ are iid.

You pointed out that $$E[D_i] = \pi_d\bar \mu_1$$ therefore, by linearity of expectation, we should expect $$N\pi_d\bar \mu_1$$ combined drop rate.

The steady state distribution of combined drop rate would be the discrete convolution of $$N$$ copies of $$\pi_d$$

• minor input, $\pi_d\bar \mu_1$ is the drop probability derived from Markov analysis. Is it ok to understand it as an expected value? Aug 5 at 8:59
• @YuktiKaura -- I interpreted it as an average drop rate because $\pi_d$ is a steady-state probability (over many packets) for one device. Over a large number of packets we find the device spends $\pi_d$ percent of its time in the final state, and of the times its in the final state, it will fail again $\bar \mu_1$ percent of the time, so the packet will be dropped. So, if I have several devices, each getting the same packet rate and having same behavior, then each on, on average will also drop at rate $\pi_d \bar \mu_1$ Aug 5 at 19:30
• Thank you... Got it, but when we multiply the expected value with $N$ the drop rate may no longer stay within the range [0,1] Aug 6 at 13:11
• @YuktiKaura since it's a rate I wasn't expecting that to be an issue -- if you have 1000 devices they will drop $1000\pi_d\bar \mu_1$ packets per unit time. Or...are you asking for the probability that ANY of the devices will drop something (like in reliability engineering) in which case it would be something like $1-(\pi_d\mu_1)^N$ Aug 6 at 13:33
• Precisely, the formulation $N\pi_d\mu_1$ would give us the expected number of packets dropped across total $N$ devices. Drop rate would then be the ratio of number of packets dropped (from the formulation above) to the number of packets transmitted ain't it? Aug 6 at 14:05