# Confidence Intervals: Sampling Distribution of the Sample Mean or the Distribution itself?

When I see a confidence interval, such as the Z-Interval, is it approximating the sampling distribution of sample means or approximating a normal distribution from the sample or is it something else entirely? In particular, I'm wondering about Garwood’s CI for the Poisson Distribution.

• Googling around, I find three somewhat different specifications of 'Garwood CIs'. If you want a specific answer, then it would be helpful show the type of CI you are asking about. – BruceET Apr 16 '17 at 19:37
• @BruceET - the Garwood CI can be written different ways but it generally refers to F Garwood's Fiducial Limit for the Poisson distribution, which he wrote about in the 1936 article of Biometrika. The paper: jstor.org/stable/2333958 The interval: danielsoper.com/statcalc/formulas.aspx?id=86 – Matthew Anderson Apr 16 '17 at 21:26

Confidence intervals (CI) are often based on a pivotal quantity. Such a quantity is a function of data and unobservable parameters, with a distribution that does not depend on the unobservable parameters.

For example, a t CI for the mean $\mu$ of a normal distribution is based on $T = \frac{\bar X - \mu}{S/\sqrt{n}} \sim \mathsf{T}(n-1).$ Thus a 95% CI results from $$P(L \le T \le U) = P\left(\bar X - U\frac{S}{\sqrt{n}} \le \mu \le \bar X - L\frac{S}{\sqrt{n}}\right) = .95,$$ where $L$ and $U$ cut 2.5% of the area from the lower and upper tails of $\mathsf{T}(n-1),$ respectively. Because of the symmetry of the t distribution, this can be written as $\bar X \pm t^*S/\sqrt{n},$ for appropriate $t^*.$ Here $S/\sqrt{n}$ is often called the (estimated) standard error of the point estimate $\bar X$ and $t^*S/\sqrt{n}$ is called the margin of error of the CI.

Similarly, in estimating the normal variance the quantity $Q = \frac{(n-1)S^2}{\sigma^2} \sim \mathsf{Chisq}(n-1)$ is used to obtain the CI $\left(\frac{(n-1)S^2}{U}, \frac{(n-2)S^2}{L}\right)$ as a CI for $\sigma^2.$ However, because $\mathsf{Chisq}(n-1)$ is not symmetrical, this CI is not centered at $S^2$ and the terminologies 'standard error' and 'margin of error' are not used in the same way.

The two types of CIs mentioned above are sometimes said to arise from "inverting the test" of $H_0: \mu = \mu_0$ vs. $H_a: \mu \ne \mu_0$ or the test of $H_0: \sigma^2 = \sigma^2_0$ vs. $H_a: \sigma^2 \ne \sigma^2_0,$ at level $\alpha$ where "acceptable" values $\mu_0$ or $\sigma_0^2,$ respectively, yield $(1-\alpha)\%$ CIs.

Commonly used 95% CIs for Poisson $\lambda,$ based on $X$ observances of a Poisson event, result from attempts to invert the approximate normal test, where one rejects $H_0: \lambda = \lambda_0$ against the two-sided alternative, when $|Z| = \frac{|X - \lambda_0|}{\sqrt{\lambda_0}} > 1.96.$ These CIs are based on the approximate standard normality of $Z$.

The Wald interval $\bar X \pm 1.96\sqrt{X}$ results from using $X$ for $\lambda,$ which is OK for large $\lambda$ and not so good for small $\lambda.$

A more sophisticated inversion of the asymptotic test (solving a quadratic inequality and conflating 2's with 1.96's) results in the 95% CI $X + 2 \pm 1.96\sqrt{X + 1}.$ It goes by various names and is somewhat analagous to the Agresti (plus-4) CI for binomial success probability $\theta.$ It has actual coverage probabilities near 95%, roughly for $\lambda > 5.$

One sometimes uses the terminologies 'standard error' and 'margin of error' for these Poisson CIs because they are based on approximations to the symmetrical normal distribution. Both of them ignore the asymmetry of the Poisson distribution, marked when $\lambda$ is small.

There are many 'exact' types of Poisson CIs that attempt to provide at least the promised coverage probabilties (at least for one-sided CIs). They work better for small $\lambda,$ but are often too long to be useful in practice. I have not gone through the derivation, and I am not sure whether the Garwood interval is in this class of intervals.

For $X = 8,$ the interval $X + 2 \pm 1.96(X + 1)$ gives $10 \pm 5.88$ or $(4.12, 15.88).$ The Garwood interval gives $(3.45, 14.42).$ So the difference is not great in this particular case.

qchisq(.025, 16)/2
## 3.453832
qchisq(.975, 16)/2
## 14.42268


Here are coverage probabilities [$\lambda \in (.3, 40)$] for these two kinds of CIs--'target coverage' 95%. I have called the former 'Agresti' even though that is not established nomenclature.

Modified, per Comment. The Garwood interval is quite conservative (often longer than necessary), and has coverage probability often considerably exceeding 95%. (My R code is pasted below the plots.)

par(mfrow=c(1,2))
## Agresti
lam = seq(.3, 40, by=.0001);  m = length(lam)       # values of lambda
t = 0:200                                           # realistic values of T
LL = t+2 - 1.96*sqrt(t+1); UL = t+2 + 1.96*sqrt(t+1)  # corresp. CIs
cov.pr = numeric(m)
for(i in 1:m) {
lam.i = lam[i]                              # pick a lambda
cov = (lam.i >= LL & lam.i <= UL)           # TRUE if CI covers
cov.pr[i] = sum(dpois(t[cov], lam.i)) }     # sum probs for covering T's
plot(lam, cov.pr, type="l", xlim=c(0,40), ylim=c(.8,1), lwd=2, xaxs="i", main="Agresti")
abline(h=.95, col="green3")
## Garwood
lam = seq(.3, 40, by=.0001);  m = length(lam)         # values of lambda
t = 0:200                                             # realistic values of T
LL = qchisq(.025, 2*t)/2; UL = qchisq(.975, 2*t=2)/2  # corresp. CIs
cov.pr = numeric(m)
for(i in 1:m) {
lam.i = lam[i]                              # pick a lambda
cov = (lam.i >= LL & lam.i <= UL)           # TRUE if CI covers
cov.pr[i] = sum(dpois(t[cov], lam.i)) }     # sum probs for covering T's
plot(lam, cov.pr, type="l", xlim=c(0,40), ylim=c(.8,1), lwd=2, xaxs="i", main="Garwood")
abline(h=.95, col="green3")
par(mfrow=c(1,1))

• I believe that the Garwood CI is an exact CI and is sometimes too wide, but nonetheless I was wondering about it. However, when reading this paper I did notice that in the interval (4,50], the Garwood CI is among the narrowest intervals. – Matthew Anderson Apr 16 '17 at 23:09
• Hey, I was reading and testing your R code and something looks off. For one, this: LL = qchisq(.025, 2*t)/2; UL = qchisq(.975, 2*t)/2 should actually be LL = qchisq(.025, 2*t)/2; UL = qchisq(.975, 2*t+2)/2. The upper boundary uses 2t+2, not just 2t. This will modify your graph by a lot. – Matthew Anderson Apr 20 '17 at 2:04
• I saw a couple of alleged versions of Garwood's CI. Your version makes sense, in view of comments I saw about it being 'superconservative'. If you want $2t+t$ at the upper boundary, you shall have it. I modified the code and figure accordingly. Thanks for your comment. – BruceET Apr 20 '17 at 4:53

You want to compute confidence intervals for the $\lambda$ parameter of a Poisson distribution? What does that mean. Well, you want some procedure that takes a random sample $x_1\ldots x_n$ from the Poisson distribution of unknown $\lambda$ and produces and inteveral. You want it to be the case that such an interval constructed from the data has a $95\%$ chance of covering the true value of $\lambda$ (averaged over samples, imagining collecting a sample of data to be an independent trial of sorts. You want a $95\%$ success probability.)

The most common way to do this is to construct an estimator for $\lambda$ and then compute its sampling distribution. Since you know $\lambda$ is the mean of the Poisson distribution one might expect $$\hat\lambda = (x_1+x_2+\ldots +x_n)/n$$ to be a pretty good estimator for it and it is, so we'll use it.

Now you want to compute $\hat\lambda$'s sampling distribution (which is a function of $\lambda$ of course). Then for any given true $\lambda$ you can compute the probability that $\hat\lambda$ will be more than a certain distance away from it. In so doing you can come up with an interval around $\lambda$ that $\hat\lambda$ would fall into 95% of the time. This is not the confidence interval. Remember we don't know $\lambda$... we want an interval around $\hat\lambda.$

But of course we can turn it around. For each value of the estimator $\hat\lambda$ we can collect all values of $\lambda$ for which $\hat\lambda$ falls into that interval we constructed around it last paragraph (that interval that wasn't the confidence interval). This collection of $\lambda$'s is the CI about $\hat\lambda$.

So we see that a sampling distribution for $\hat\lambda$ leads to a confidence interval for $\lambda$, although it's a bit more complicated than just being a quantile of the sampling distribution.

Now, the hard part is actually coming up with a good procedure for computing the sampling distribution (and also there's some ambiguity in how we choose the interval... is it best to make it symmetric or should we skew it since the Poisson distribution is skewed?) To do it exactly, we would need to compute the distribution of $(X_1+X_2+\ldots X_n)/n$ where $X_i$ are independent Poissons w/ parameter $\lambda.$ If the sample size $n$ is large, then the central limit theorem will guarantee our estimator is close to normal, but that's true of pretty much any case so is boring (This is why $Z$ statistics are said to be ok for large samples). Even in small sample, in the event that you get a very large $\hat\lambda \gg 1,$ it's probably the case that the Poisson distribution is nearly normal so a normal distribution would probably work well here, but not when $\hat\lambda$ is small. I'm sure Garwood's method finds some clever way to handle the more general case.

• I believe Garwood's uses the chi-square distribution to approximate the sampling distribution of a poisson random variable but I wasn't sure. This answer kind of clears the air, and thank you for a good response. – Matthew Anderson Apr 16 '17 at 21:28
• @MatthewAnderson Yes, I was unfamiliar, but that seems plausible since the chi squared is a sum of skewed positive RVs. I was trying to emphasize that the CI isn't a quantile of the sampling distribution (but that the sampling dist is integral to its construction). Since the Garwood article has the word 'fiducial' in the title it I'm guessing it obscures this point. – spaceisdarkgreen Apr 16 '17 at 21:34