How to calculate a confidence interval for $p$, the parameter of the binomial distribution? Suppose I have a random variable X that is known to follow the binomial distribution B, but whose parameter $p$ is unknown.
I have observed 100 samples of X, and they all came out true.
How can I calculate a 95% confidence interval for the value of $p$ (which stands for P(true)) of the underlying distribution B?
 A: First of all, the usual use of the notation $\hat{p}$ refers to the (point) estimate and not the true underlying value of the parameter. Many answers to your question are available in standard texts. 
The Clopper–Pearson interval is the most popular. It is guaranteed to give at least 95% coverage for all values of the underlying parameter. It is actually more stringent, in that it ensures that there is no more than 2.5% non-coverage in each tail (i.e. it is a "central" interval). As a consequence it tends to be quite conservative. That said, the typical "approximate intervals" (e.g., Wald and score) both fail miserably if the true proportion is close to 0 or 1.
Another exact method which is nested within (and usually narrower than) the Clopper–Pearson interval can be found in

H. Blaker, Confidence curves and improved exact confidence intervals for discrete distributions, Canadian J. Stat., vol 28, 2000, 783–798.

Unlike the Copper–Pearson interval, there is no closed form for the Blaker interval, but software is available.
Agresti also covers this in Section 1.4.4 (pp. 18–21) of Categorical Data Analysis, 2nd. edition. (This is the larger green book, not the smaller one of almost the same name.)
For 100 successes in 100 trials, the Clopper–Pearson confidence interval is $(0.9638, 1.0)$ and the Blaker confidence interval is $(0.9644, 1.0)$.
