Consider Markov process, where a person is searching for some particular piece of information among the list of all kinds of data. The first time the person studies some new piece of information, the probability that it satisfies the person is A. If this piece satisfies the person, then the person ceases his search. Otherwise, the person continues. The probability that the person is satisfied by the second result is also A and so on. For simplicity, we may assume that there is an infinite list of pieces of information, so theoretically, the person may never reach the end. The questions asks to calculate expectation of the number of results the person studies and its variance.

I have started solving the problem, but all I can think about is that it is just a Geometric distribution. I do not really see where I can use the property of the Markov process. I understand that maybe the pieces of information are not independent, but I don’t see how can I apply this to the problem to get any meaningful result.

  • $\begingroup$ That just sounds like a geometric random variable, which technically can be represented by a two state Markov process but that's overkill. $\endgroup$ – Ian Mar 13 at 19:53
  • $\begingroup$ Yes, this sounds exactly like the Geometric distribution with probability of success A. However, this question was given on the test for stochastic processes class. The professor made it explicit that we had to use some properties of Markov process in each question. $\endgroup$ – Renat Sergazinov Mar 13 at 19:57
  • 1
    $\begingroup$ Fair; then this is a two state process with probability A to go from 1 to 2 and probability 1 to go from 2 to 2. And you want to calculate moments of the absorption time, which can be done using renewal theory (even though this is overkill here). $\endgroup$ – Ian Mar 13 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.