# Expected number of users reaching a particular state

I have a problem to solve where N users chooses value with uniform distribution between 3 and 7 initially. Then every second, every user decrements its value (like if 7 is chosen 7, then it becomes 6, if 3, then it becomes 2).

I want to calculate expected number of users reaching state when they have the value of 1.

I have solved for getting probability $$P_1$$ to be in state with value of 1 using Markov chain, but how can I know the number of users every second that reach the value of 1 ? After getting to the value of 1 each user again chooses a number between 3 and 7 randomly and process continues.

• If $p_1$ is the probability in the stationary distribution then perhaps you are looking for $Np_1$ Aug 22, 2020 at 15:36
• Are the values chosen integer values? Aug 22, 2020 at 15:54
• yes integer values Aug 22, 2020 at 16:09
• @Henry Thanks for the answer, would it make difference that after reaching value of 1 by every user they again repeat process of again choosing value between 3 and 7 and then again they reach 1. So would {Np_1} remain same? Aug 22, 2020 at 16:13
• I'm confused. If they all start out between $3$ and $7$, and they each decrement by one per second, don't they all eventually reach $1$? I'm obviously not understanding this question. Aug 22, 2020 at 16:54

exactly what it is that you want to calculate. Given the scenario you have described, however, the state of every user can be represented by a $$7$$-state Markov chain with transition matrix $$P=\pmatrix{0&0&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}\\ 1&0&0&0&0&0&0\\ 0&1&0&0&0&0&0\\ 0&0&1&0&0&0&0\\ 0&0&0&1&0&0&0\\ 0&0&0&0&1&0&0\\ 0&0&0&0&0&1&0}\ .$$ and initial distribution $$\pi_1=\pmatrix{0&0&\frac{1}{5} &\frac{1}{5} &\frac{1}{5} &\frac{1}{5} &\frac{1}{5}}\ .$$ The distribution $$\ \pi_t\$$ of a user's state at time $$\ t\$$ is given by $$\pi_t=\pi_1P^{t-1}\ .$$ If there are $$\ N\$$ users at the start, then the expected number $$\ e_{tj}\$$ of users in state $$\ j\$$ at time $$\ t\$$ is given by $$e_{tj}=N\pi_{tj}\ .$$ If you want an explicit formula for $$\ \pi_{t1}\$$, you can get it in terms of the eigenvalues of $$\ P\$$, which are the roots of its characteristic equation: $$x^7-\frac{1}{5}\left(x^4+x^3+x^2+x+1\right)=0\ .$$ The stationary distribution of the chain is $$\pi_\infty=\pmatrix{\frac{1}{5} &\frac{1}{5} &\frac{1}{5} &\frac{4}{25} &\frac{3}{25} &\frac{2}{25} &\frac{1}{25}}\ ,$$ so for sufficiently large $$\ t\$$, the expected number of users in sate $$\ 1\$$ will be $$N\pi_{t1}\approx \frac{N}{5}\ .$$
• I had inadvertently reversed the entries in the stationary distribution, so my original answer was incorrect. I've now corrected it (I hope). The expression for the stationary distribution $\ \pi_\infty\$ at least now does satisfy the equation $\ \pi_\infty P= \pi_\infty\$, which it didn't do before I corrected it. Aug 26, 2020 at 12:13