# Definitional Question about Confidence Intervals

My Statistics Homework went as follows

"A statistics professor asked her students whether or not they were registered to vote. In a sample of 50 of her students (randomly sampled from her 700 students), 35 said they were registered to vote."

After asking you to make a 95% confidence interval, it then asks you this:

"What is the probability that the true proportion of the professor's students who were registered to vote is in your confidence interval?"

My Stat teacher says this is a trick question and there is no probability. I, however think there is a probability. Who is right here?

This is a subtler question than perhaps the teacher realizes, and it's actually a fairly standard thing. If you assign a prior probability distribution to the proportion registered to vote and then find a $95\%$ conditional probability interval given your $35\%$ of $50$ data, then you can say the probability is $95\%$. That is a biased interval -- specifically it is biased in favor of percentages that are more probable under the prior distribution.

Now the hard question. Should one assign probabilities

• only to things that are random? If you take a new random sample of $50$ independent of the first one, some things change, and those are the things we call "random". Or$\ldots$
• to all things that are uncertain? That there was life on Mars a billion years ago is uncertain and one might assign it a probability of $1/2$, but it's not "random": it will not change if we take a new sample independent of the first sample (e.g. create a new Universe identical to this one, but deciding at random independently of this one, whether there will have been life on Mars a billion years ago in that particular Creation). Nor can we say there was life on Mars a billion years ago in $1/2$ of all Universes that are otherwise just like this one.

The first bullet above is frequentism and the second is Bayesianism.

Now to your voter-registration question. The percentage of the $700$ students who are registered to vote is not random, in that it will not change if a new sample is taken. Under frequentism, one would therefore not assign it any probability distribution, or else one would say that the probability is $0$ or $1$ that the percentange is between this much and this much, but we don't know whether it's $0$ or $1$. And that is just what many teachers of statistics do say about that, and I won't be surprised if your teacher says that. Under Bayesianism, one can say something. But here people complain that the initial, or prior, probability distribution of the percentage is subjective (but the amount of subjectivity may be allowed by some critics to go away as the sample size increases).

In either regime -- frequentist or Bayesian -- the probabilty that the confidence interval yielded by the next independent sample that you take will contain the true percentage is $95\%$.

In practice people behave as if the $95\%$ is how epistemically sure one should be after one computes the confidence interval. Under some circumstances, I would argue that that makes sense. In particular, I do not know what data someone got when they took a random sample from this population on April 1, 2015. What is the conditional probability, given my current knowledge that the confidence interval that that person gets contains the true percent? It is $0.95$, regardless of the position one takes on Bayesianism versus frequentism.

Here's the catch: the true proportion is either in the confidence interval or it isn't. That is, you sampled 50 of them, 35 of them said that they are registered, so you get a confidence interval of (for illustration) $(0.6,0.7)$ for the proportion that are registered in the entire class. Now you can imagine actually asking the entire class, and you'll get a number. At this point nothing is random, and nothing is uncertain: your interval is a fixed, specified interval, and the true proportion is a fixed, known number. So the true proportion is either in the confidence interval or it isn't.

Although the true proportion is a fixed number, the confidence interval is random. The precise sense in which the confidence interval is a 95% confidence interval is that 95% of the random intervals generated by samples will contain the true proportion (provided the relevant model assumptions hold). The sample we actually took might or might not be one of the accurate ones.

The idea that we are "confident" that the true proportion is in our given interval only makes sense in the Bayesian framework. But Michael Hardy has already touched on this, so I'll leave that discussion to him.