Suppose there exists this special coin with positive reenforcement properties. Initially, this coin lands heads with probability $p$. However, the special property is that for every time the coin lands heads, the probability that the next toss is heads goes up by $w$ such that the new heads probability is $p+w$. If the coin lands tails, the opposite happens and the new probability that the coin lands heads is $p-w$. Hence, the probability the coin lands heads on the ith toss is $p_i=p_{i-1}+wX_{i-1}$, where $X_{i-1}$ is a random variable that takes value $1$ if the coin lands heads on the $i$-1th flip, and $-1$ if the coin lands tails on the $i$-1th flip.

Now consider the random variable: $$X=\sum_{n=1}^{\infty}X_i$$

Whats $\mathbb{E}{X}$?

This problem seems similar to one i've seen before wherein you are picking balls from a bag, and you add back more balls of the same color that you pulled out on the ith pull. However, it seems to me that this answer may depend on the magnitude of $w$ and the initial probability of the coin coming up heads.

  • 2
    $\begingroup$ In this formulation the probability $p$ will end up greater than $1$ or less than $0$. You have to specify a different way to specify the reinforcement. $\endgroup$ – Ethan Bolker Mar 3 '17 at 0:37
  • $\begingroup$ Well if the probability goes less than 0 than greater or 1, wouldn't every $X_i$ be determined from there on out? All heads or all tails is guaranteed after passing that threshold right? $\endgroup$ – Leif Ericson Mar 3 '17 at 0:46
  • $\begingroup$ Yes. If that's OK with you leave the question as is. Mayve someone will answer. My intuition is that with probability $1$ the process will eventually produce all tails or all heads, I've nothing else to contribute. $\endgroup$ – Ethan Bolker Mar 3 '17 at 0:52
  • $\begingroup$ @EthanBolker: And if $w = 1,$ 'eventually' comes very soon. Maybe a more interesting Markov Chain if $p_i = (p_{i-1} + X_{i-1})/2.$ $\endgroup$ – BruceET Mar 3 '17 at 1:18
  • $\begingroup$ The essential problem with the way your question is formulated is that, with nonzero probability, if $p > 0$ and $X_i = 1$ if the $i^{\rm th}$ toss is heads, then $\Pr[X = \infty] > 0$, hence $\operatorname{E}[X] = \infty$ trivially. $\endgroup$ – heropup Mar 3 '17 at 1:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.