Here's a problem I thought of that I don't know how to approach:

You have a fair coin that you keep on flipping. After every flip, you perform a hypothesis test based on all coin flips thus far, with significance level $\alpha$, where your null hypothesis is that the coin is fair and your alternative hypothesis is that the coin is not fair. In terms of $\alpha$, what is the expected number of flips before the first time that you reject the null hypothesis?

Edit based on comment below: For what values of $\alpha$ is the answer to the question above finite? For those values for which it is infinite, what is the probability that the null hypothesis will ever be rejected, in terms of $\alpha$?

Edit 2: My post was edited to say "You believe that you have a fair coin." The coin is in fact fair, and you know that. You do the hypothesis tests anyway. Otherwise the problem is unapproachable because you don't know the probability that any particular toss will come up a certain way.

  • $\begingroup$ If there is a positive probability that you never reject the null hypothesis, then the expectation is infinite. [I don't know whether or not there is such a positive probability.] $\endgroup$ – paw88789 May 9 '15 at 12:41

EDIT: This answer was unclear for OP at first, so I tried to make it clearer through a new approach. Apparently it arose another legitimate doubt, so I tried now to put both answers together and clarify them even more. (Still I might be wrong, but I'll try to express myself better)

What you look for, is the expected number of tosses before we do a Type I error (rejecting $H_0$ when it was true). The probability of that is precisely $\alpha$ (that's another way to define it).

So $P(Type\ I\ error)=\alpha$.

Let $X_n$ be the event of rejecting $n^{th}$ test.

Now, $E[X_1]=\alpha$ stands for the expected number of games (a game is starting to test in the way we do a new coin) where $H_0$ was rejected on the first throw. $E[X_1+X_2]=E[X_1]+E[X_2]$ is the expected number of games where $H_0$ is rejected either on the first or the second throw. Note that with most $\alpha$ this will be lower than $1$, so the expectation for a single game is not to reject $H_0$ yet.

When do we expect to have rejected $H_0$? Precisely when the number of expected games in which we reject $H_0$ is $1$. Therefore, we look for $n$ such as $$ E[X_1+X_2+...+X_n]=1\\ E[X_1+X_2+...+X_n]=E[nX_1]=nE[X_1]=n\alpha=1\\ n=\frac{1}{\alpha} $$

The other answer goes like this: Let the variable $T$ count the number of tests before rejecting one. We look for $E[T]$.

Also, using previous notation, $P(X_n)=\alpha(1-\alpha)^{n-1}$ (I'm aware this implies independence between the events $X_n$ and $X_{n-1}$ but since I'm looking for the expected value, for the linearity of the Expected Value, it shouldn't be a problem, though I'm aware I'm not being polite with notation).

$$E[T] = \sum_{n=1}^{\infty}nP(X_n) = \sum_{n=1}^{\infty}n\alpha(1-\alpha)^{n-1}= \alpha\sum_{n=1}^{\infty}n(1-\alpha)^{n-1}=\\ \alpha\sum_{n=0}^{\infty}(n+1)(1-\alpha)^{n}= \alpha(\sum_{n=0}^{\infty}n(1-\alpha)^{n}+\sum_{n=0}^{\infty}(1-\alpha)^{n}) = \alpha(\frac{1-\alpha}{\alpha^2}+\frac{1}{\alpha}) \\ E[T]=\frac{1}{\alpha} $$

  • $\begingroup$ Can you explain why we are looking for $n$ such that $E[X_1 + X_2 + \dots + X_n] = 1$? $\endgroup$ – Eric Neyman May 23 '15 at 18:28
  • $\begingroup$ I look for when the expected number of rejected $H_0$ is 1. $\endgroup$ – Masclins May 23 '15 at 18:32
  • $\begingroup$ Why is this the same as the expected number of flips before the null hypothesis is rejected for the first time? $\endgroup$ – Eric Neyman May 24 '15 at 20:28
  • $\begingroup$ It's what you asked. The expected number of flips until a hypothesis test rejects $H_0$, when doing one after each flip. I don't get your doubt, please be more specific about what you don't understand of the reasoning. $\endgroup$ – Masclins May 24 '15 at 20:36
  • $\begingroup$ You showed that $\frac{1}{\alpha}$ is the value of $n$ such that the expected number of rejections for the first $n$ tests is equal to $1$. I am asking for the expected number of tests before the first rejection. I may be missing something, but I don't see why those two should be the same. $\endgroup$ – Eric Neyman May 25 '15 at 21:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.