I have a problem to solve where N users chooses value with uniform distribution between 3 and 7 initially. Then every second, every user decrements its value (like if 7 is chosen 7, then it becomes 6, if 3, then it becomes 2).

I want to calculate expected number of users reaching state when they have the value of 1.

I have solved for getting probability $P_1$ to be in state with value of 1 using Markov chain, but how can I know the number of users every second that reach the value of 1 ? After getting to the value of 1 each user again chooses a number between 3 and 7 randomly and process continues.

  • $\begingroup$ If $p_1$ is the probability in the stationary distribution then perhaps you are looking for $Np_1$ $\endgroup$
    – Henry
    Aug 22, 2020 at 15:36
  • $\begingroup$ Are the values chosen integer values? $\endgroup$ Aug 22, 2020 at 15:54
  • $\begingroup$ yes integer values $\endgroup$ Aug 22, 2020 at 16:09
  • $\begingroup$ @Henry Thanks for the answer, would it make difference that after reaching value of 1 by every user they again repeat process of again choosing value between 3 and 7 and then again they reach 1. So would {Np_1} remain same? $\endgroup$ Aug 22, 2020 at 16:13
  • $\begingroup$ I'm confused. If they all start out between $3$ and $7$, and they each decrement by one per second, don't they all eventually reach $1$? I'm obviously not understanding this question. $\endgroup$
    – Brian Tung
    Aug 22, 2020 at 16:54

1 Answer 1


It is not at all clear from your following statement:

I want to calculate expected number of users reaching state when they have the value of 1.

exactly what it is that you want to calculate. Given the scenario you have described, however, the state of every user can be represented by a $7$-state Markov chain with transition matrix $$ P=\pmatrix{0&0&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}\\ 1&0&0&0&0&0&0\\ 0&1&0&0&0&0&0\\ 0&0&1&0&0&0&0\\ 0&0&0&1&0&0&0\\ 0&0&0&0&1&0&0\\ 0&0&0&0&0&1&0}\ . $$ and initial distribution $$ \pi_1=\pmatrix{0&0&\frac{1}{5} &\frac{1}{5} &\frac{1}{5} &\frac{1}{5} &\frac{1}{5}}\ . $$ The distribution $\ \pi_t\ $ of a user's state at time $\ t\ $ is given by $$ \pi_t=\pi_1P^{t-1}\ . $$ If there are $\ N\ $ users at the start, then the expected number $\ e_{tj}\ $ of users in state $\ j\ $ at time $\ t\ $ is given by $$ e_{tj}=N\pi_{tj}\ . $$ If you want an explicit formula for $\ \pi_{t1}\ $, you can get it in terms of the eigenvalues of $\ P\ $, which are the roots of its characteristic equation: $$ x^7-\frac{1}{5}\left(x^4+x^3+x^2+x+1\right)=0\ . $$ The stationary distribution of the chain is $$ \pi_\infty=\pmatrix{\frac{1}{5} &\frac{1}{5} &\frac{1}{5} &\frac{4}{25} &\frac{3}{25} &\frac{2}{25} &\frac{1}{25}}\ , $$ so for sufficiently large $\ t\ $, the expected number of users in sate $\ 1\ $ will be $$ N\pi_{t1}\approx \frac{N}{5}\ . $$

  • $\begingroup$ CAn any one tell which answer is correct either @Henry or @ lonza , since results from both are different $\endgroup$ Aug 26, 2020 at 5:40
  • $\begingroup$ I had inadvertently reversed the entries in the stationary distribution, so my original answer was incorrect. I've now corrected it (I hope). The expression for the stationary distribution $\ \pi_\infty\ $ at least now does satisfy the equation $\ \pi_\infty P= \pi_\infty\ $, which it didn't do before I corrected it. $\endgroup$ Aug 26, 2020 at 12:13
  • $\begingroup$ Thanks @Henry and lonza. $\endgroup$ Aug 27, 2020 at 19:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .