# What does level of confidence mean in statistics?

The thing I am talking about is the interval estimate of the population mean. Let $$X\sim N(\mu,\sigma^2)$$ where $$\mu$$ is unknown but $$\sigma$$ is known. To estimate $$\mu$$, we perform $$n$$ experiments and find the mean $$\overline X$$. On my statistics book, there is something like this:

Let $$Z=\frac{\overline X-\mu}{\sigma/\sqrt n}$$ so $$P(-r where $$\alpha$$ is the given confidence level. Of course, we should choose $$r=\Phi^{-1}(\frac{1+\alpha}{2})$$.

The next step is what makes me worried.

Therefore, if in one experiment we get $$\overline X= \bar x$$, then $$P(\overline x-r\sigma/\sqrt n<\mu<\overline x+r\sigma/\sqrt n) =\alpha \text{ }(*)$$

To me, this is very strange: it's just like having a random variable $$Y$$ with $$P(Y<3)=0.95$$, and in an experiment, we observe that $$Y=3.5$$, so we replace $$Y$$ with the observed value, and write $$P(3.5<3)=0.95$$, which is ridiculous.

What $$(*)$$ is doing is to give a probability of $$\mu$$ lying in a certain interval - but to give $$\mu$$ a probability, $$\mu$$ must have a distribution first. So, what is the distribution of $$\mu$$?

• It’s the probability that the interval (which is random since it depends on the sample $\bar x$) will cover $\mu$ (which as you’ve noted is an unknown constant and not random). – spaceisdarkgreen Oct 18 '19 at 0:07
• One resolution (not without faults) is to interpret it as a statement about where the RV $\bar{x}$ lies. With chance $\alpha$ it’ll be so-and-so within the true value of $\mu$. If I recall correctly there are many questions on this site discussing the interpretations of CI and, even better, on the stats stack exchange, (as well as on the wikipedia page for CI). – Nap D. Lover Oct 18 '19 at 0:08
• The "next step" you quoted looks wrong to me too. If it had $\bar X$ (a random variable) in place of $\bar x$ (a number) it would be ok. – littleO Oct 18 '19 at 0:51
• Which statistics book is this from? (And what page?) – littleO Oct 18 '19 at 5:23
• $[\overline x-r\sigma/\sqrt n,\overline x+r\sigma/\sqrt n]$ is not a random interval as opposed to $[\overline X-r\sigma/\sqrt n, \overline X-r\sigma/\sqrt n]$, and so $$P(\overline x-r\sigma/\sqrt n<\mu<\overline x+r\sigma/\sqrt n)=0 \text{ or } 1.$$ – d.k.o. Oct 18 '19 at 12:48

If $$\mu$$ is not known and $$\sigma$$ is known, then a 95% confidence interval for $$\mu$$ is of the form $$\bar X \pm 1.96\sigma/\sqrt{n}.$$

For a normal sample, $$\bar X$$ is the best estimate of $$\mu,$$ but it is not perfect. It is subject to variability.

• First because there is variability in the population, expressed by $$\sigma.$$ Second, because each sample is is different.

• However, large samples can overcome population variability. For example, the population standard deviation $$\sigma = 25$$ might be a huge population variability.

• Specifically, if you have $$n = 100$$ observations in your sample, it is possible to show that the standard deviation of $$\bar X$$ is decreased to $$25/\sqrt{100} = 25/10 = 2.5,$$ which might not be so bad.

• The confidence interval extends about two standard deviations (relative to $$\bar X)$$ on either side of $$\bar X.$$ Thus, the confidence has a 'margin of error' to account for this remaining variability. (Here 'about two' turns out to be $$1.96.)$$

There can be no guarantee that any one 95% confidence interval (CI) truly includes the value of the population mean $$\mu.$$ But if you repeatedly use the formula in my first paragraph, then over the long run 95% of your CIs will include $$\mu.$$

Example: Suppose that, unknown to you, $$\mu = 200$$ and that, known to you $$\sigma = 25.$$ Let's make 20 CIs by taking $$n - 100$$ observations for this population---for each CI. (That's 2000 observations altogether, but I'm sampling by computer so it's easy to do.)

In the figure below each of the 20 vertical bars represents a CI made with the formula above. Dots are sample means. The horizontal line at $$\mu=200$$ makes it is easy to see which CIs contain $$\mu.$$ In particular, We can see that Sample 17 yielded a CI that does not include $$\mu.$$

Notes: (1) This is a simulation so (in the background) we know $$\mu=200,$$ but in real life experiments, $$\mu$$ wouldn't be exactly known.

(2) Here is R code used to make the figure.

set.seed(1234)
m = 20;  a = lcl = ucl = numeric(m)
mu = 200; sg = 25; n = 100;  me=1.96*sg/sqrt(n)
for(i in 1:m) {
x = rnorm(n, mu, sg)
a[i] = mean(x)
lcl[i] = a[i] - me;  ucl[i] = a[i] + me }
top = max(ucl);  bot = min(lcl)
plot(c(1,20), c(top,bot), col="white",
ylab="Units", xlab="Sample",
main="CIs From 20 Normal Sample of Size 100")
for(i in 1:m) {
lines(c(i,i), c(lcl[i], ucl[i]))
points(i, a[i], pch=19, col="blue") }
abline(h=mu, col="green3")

• I'm not sure why this got downvoted... it looks correct to me – spaceisdarkgreen Oct 18 '19 at 1:48
• (I mean, it's a little convenient that exactly 1/20 simulations didn't cover, but that's the only thing that seems shady here :) ) – spaceisdarkgreen Oct 18 '19 at 1:55
• I am not asking about this... I am asking about the interpretation of probability on $\mu$. – Jethro Oct 18 '19 at 2:04
• @Jethro You are asking about how to interpret confidence intervals and this is the answer. There is no notion of a probability distribution for $\mu$... it is a constant. In a Bayesian approach there is a notion of a distribution for $\mu$ that quantifies in some sense our belief and uncertainty about it (given an assumed prior distribution), but confidence intervals are a frequentist method. The Bayesian version are called credible intervals. – spaceisdarkgreen Oct 18 '19 at 3:16
• @Jethro. Tried to show you the standard frequentist interpretation, which is long run fraction covering. Strictly speaking, either $\mu$ is in CI or not. No probability about it. If you want Bayesian interpretation of posterior probability interval or 'credible' interval, then ask that question. – BruceET Oct 18 '19 at 5:08

Refer to Wikipedia:

Various interpretations of a confidence interval can be given (taking the 90% confidence interval as an example in the following).

• The confidence interval can be expressed in terms of samples (or repeated samples): "Were this procedure to be repeated on numerous samples, the fraction of calculated confidence intervals (which would differ for each sample) that encompass the true population parameter would tend toward 90%."
• The confidence interval can be expressed in terms of a single sample: "There is a 90% probability that the calculated confidence interval from some future experiment encompasses the true value of the population parameter." Note this is a probability statement about the confidence interval, not the population parameter. This considers the probability associated with a confidence interval from a pre-experiment point of view, in the same context in which arguments for the random allocation of treatments to study items are made. Here the experimenter sets out the way in which they intend to calculate a confidence interval and to know, before they do the actual experiment, that the interval they will end up calculating has a particular chance of covering the true but unknown value. This is very similar to the "repeated sample" interpretation above, except that it avoids relying on considering hypothetical repeats of a sampling procedure that may not be repeatable in any meaningful sense.

• The explanation of a confidence interval can amount to something like: "The confidence interval represents values for the population parameter for which the difference between the parameter and the observed estimate is not statistically significant at the 10% level". In fact, this relates to one particular way in which a confidence interval may be constructed.

Based on the point 2, you must not consider the probability of the population parameter $$\mu$$ (because it is a constant value), but consider the probability of the confidence interval that it will include the population parameter.