Confidence Intervals: Sampling Distribution of the Sample Mean or the Distribution itself? When I see a confidence interval, such as the Z-Interval, is it approximating the sampling distribution of sample means or approximating a normal distribution from the sample or is it something else entirely? In particular, I'm wondering about Garwood’s CI for the Poisson Distribution.
 A: You want to compute confidence intervals for the $\lambda$ parameter of a Poisson distribution? What does that mean. Well, you want some procedure that takes a random sample $x_1\ldots x_n$ from the Poisson distribution of unknown $\lambda$ and produces and inteveral. You want it to be the case that such an interval constructed from the data has a $95\%$ chance of covering the true value of $\lambda$ (averaged over samples, imagining collecting a sample of data to be an independent trial of sorts. You want a $95\%$ success probability.)
The most common way to do this is to construct an estimator for $\lambda$ and then compute its sampling distribution. Since you know $\lambda$ is the mean of the Poisson distribution one might expect $$ \hat\lambda = (x_1+x_2+\ldots +x_n)/n$$
to be a pretty good estimator for it and it is, so we'll use it.
Now you want to compute $\hat\lambda$'s sampling distribution (which is a function of $\lambda$ of course). Then for any given true $\lambda$ you can compute the probability that $\hat\lambda$ will be more than a certain distance away from it. In so doing you can come up with an interval around $\lambda$ that $\hat\lambda$ would fall into 95% of the time. This is not the confidence interval. Remember we don't know $\lambda$... we want an interval around $\hat\lambda.$ 
But of course we can turn it around. For each value of the estimator $\hat\lambda$ we can collect all values of $\lambda$ for which $\hat\lambda$ falls into that interval we constructed around it last paragraph (that interval that wasn't the confidence interval). This collection of $\lambda$'s is the CI about $\hat\lambda$. 
So we see that a sampling distribution for $\hat\lambda$ leads to a confidence interval for $\lambda$, although it's a bit more complicated than just being a quantile of the sampling distribution.
Now, the hard part is actually coming up with a good procedure for computing the sampling distribution (and also there's some ambiguity in how we choose the interval... is it best to make it symmetric or should we skew it since the Poisson distribution is skewed?) To do it exactly, we would need to compute the distribution of $(X_1+X_2+\ldots X_n)/n$ where $X_i$ are independent Poissons w/ parameter $\lambda.$ If the sample size $n$ is large, then the central limit theorem will guarantee our estimator is close to normal, but that's true of pretty much any case so is boring (This is why $Z$ statistics are said to be ok for large samples). Even in small sample, in the event that you get a very large $\hat\lambda \gg 1,$ it's probably the case that the Poisson distribution is nearly normal so a normal distribution would probably work well here, but not when $\hat\lambda$ is small. I'm sure Garwood's method finds some clever way to handle the more general case.
A: Confidence intervals (CI) are often based on a pivotal quantity. Such a
quantity is a
function of data and unobservable parameters, with a distribution that
does not depend on the unobservable parameters.
For example, a t CI for the mean $\mu$ of a normal distribution is based on
$T = \frac{\bar X - \mu}{S/\sqrt{n}} \sim \mathsf{T}(n-1).$ Thus a 95% CI results from
$$P(L \le T \le U) = 
P\left(\bar X - U\frac{S}{\sqrt{n}} \le \mu \le \bar X - L\frac{S}{\sqrt{n}}\right) = .95,$$
where $L$ and $U$ cut 2.5% of the area from the lower and upper tails of 
$\mathsf{T}(n-1),$ respectively. Because of the symmetry of the t distribution,
this can be written as $\bar X \pm t^*S/\sqrt{n},$ for appropriate $t^*.$
Here $S/\sqrt{n}$ is often called the (estimated) standard error of
the point estimate $\bar X$ and $t^*S/\sqrt{n}$ is called the margin of error of the CI.
Similarly, in estimating the normal variance the
quantity $Q = \frac{(n-1)S^2}{\sigma^2} \sim \mathsf{Chisq}(n-1)$ is used
to obtain the CI $\left(\frac{(n-1)S^2}{U}, \frac{(n-2)S^2}{L}\right)$ 
as a CI for $\sigma^2.$  However, because $\mathsf{Chisq}(n-1)$ is not
symmetrical, this CI is not centered at $S^2$ and the terminologies
'standard error' and 'margin of error' are not used in the same way.
The two types of CIs mentioned above are sometimes said to arise from
"inverting the test" of $H_0: \mu = \mu_0$ vs. $H_a: \mu \ne \mu_0$
or the test of $H_0: \sigma^2 = \sigma^2_0$ vs. $H_a: \sigma^2 \ne \sigma^2_0,$
at level $\alpha$ where "acceptable" values $\mu_0$ or $\sigma_0^2,$
respectively, yield $(1-\alpha)\%$ CIs.
Commonly used 95% CIs for Poisson $\lambda,$ based on $X$ observances of
a Poisson event, result from attempts to invert the approximate normal
test, where one rejects $H_0: \lambda = \lambda_0$ against the two-sided
alternative, when $|Z| = \frac{|X - \lambda_0|}{\sqrt{\lambda_0}} > 1.96.$
These CIs are based on the approximate standard normality of $Z$.
The Wald interval $\bar X \pm 1.96\sqrt{X}$ results from using $X$ for
$\lambda,$ which is OK for large $\lambda$ and not so good for small $\lambda.$
A more sophisticated inversion of the asymptotic test (solving a quadratic inequality and conflating 2's with 1.96's) results in the
95% CI $X + 2 \pm 1.96\sqrt{X + 1}.$ It goes by various names and is
somewhat analagous to the Agresti (plus-4) CI for binomial success
probability $\theta.$ It has actual coverage probabilities near
95%, roughly for $\lambda > 5.$ 
One sometimes uses the terminologies 'standard error' and 'margin of error'
for these Poisson CIs because they are based on approximations to the
symmetrical normal distribution. Both of them ignore the asymmetry of
the Poisson distribution, marked when $\lambda$ is small. 
There are many 'exact' types of Poisson CIs that attempt to provide
at least the promised coverage probabilties (at least for one-sided CIs).
They work better for small $\lambda,$ but are often too long to be useful
in practice. I have not gone through the derivation, and I am not sure whether the Garwood interval is in this class of intervals.
For $X = 8,$ the interval $X + 2 \pm 1.96(X + 1)$ gives $10 \pm 5.88$ or
$(4.12, 15.88).$  The Garwood interval gives $(3.45, 14.42).$ So the difference
is not great in this particular case.
qchisq(.025, 16)/2
## 3.453832
qchisq(.975, 16)/2
## 14.42268

Here are coverage probabilities [$\lambda \in (.3, 40)$] for these two kinds of CIs--'target coverage' 95%. I have called
the former 'Agresti' even though that is not established nomenclature. 
Modified, per Comment. The Garwood interval is quite conservative (often
longer than necessary), and has
coverage probability often considerably exceeding 95%. (My R code is pasted below the plots.)

par(mfrow=c(1,2))
## Agresti
lam = seq(.3, 40, by=.0001);  m = length(lam)       # values of lambda
t = 0:200                                           # realistic values of T                      
LL = t+2 - 1.96*sqrt(t+1); UL = t+2 + 1.96*sqrt(t+1)  # corresp. CIs
cov.pr = numeric(m)
for(i in 1:m) {                                     
  lam.i = lam[i]                              # pick a lambda                                   
  cov = (lam.i >= LL & lam.i <= UL)           # TRUE if CI covers
  cov.pr[i] = sum(dpois(t[cov], lam.i)) }     # sum probs for covering T's
plot(lam, cov.pr, type="l", xlim=c(0,40), ylim=c(.8,1), lwd=2, xaxs="i", main="Agresti")
abline(h=.95, col="green3")
## Garwood
lam = seq(.3, 40, by=.0001);  m = length(lam)         # values of lambda
t = 0:200                                             # realistic values of T                      
LL = qchisq(.025, 2*t)/2; UL = qchisq(.975, 2*t=2)/2  # corresp. CIs
cov.pr = numeric(m)
for(i in 1:m) {                                     
  lam.i = lam[i]                              # pick a lambda                                   
  cov = (lam.i >= LL & lam.i <= UL)           # TRUE if CI covers
  cov.pr[i] = sum(dpois(t[cov], lam.i)) }     # sum probs for covering T's
plot(lam, cov.pr, type="l", xlim=c(0,40), ylim=c(.8,1), lwd=2, xaxs="i", main="Garwood")
abline(h=.95, col="green3")
par(mfrow=c(1,1))

