Confidence Interval and Hypothesis Test Disagree?

Question

I created a problem where I am having my students complete a hypothesis test and confidence interval for proportions to highlight the connection between the two. However, in doing the problem I am getting conflicting results.

With a random sample of n = 300, X = 235, and a confidence level of 99%, I obtained the following CI for the population proportion p: ( 0.72207 , 0.8446 ). I used the TI-84 calculator to get this interval and I checked using various online calculators as well. They all agreed with the interval.

The problem is set up such that the null hypothesis is Ho: p = 0.72 and the alternative hypothesis is Ha: p =/= 0.72. I am using a 1% significance level. Using the TI-84 calculator, I got a test statistic of Z = 2.443 and a P-value of 0.0145 (being 2(0.0072)). Because this P-value exceeds 0.01, I would fail to reject Ho. However, my null value of 0.72 is not within my 99% confidence interval. It is my understanding that the hypothesis test and the confidence interval must always agree as long as I have a two tailed test and the significance level and confidence level sum to 1.

Does anyone see where the contradiction occurs? Thanks much!

Answer 1

The confidence interval is $$\hat p \pm z^*_{\alpha/2} \sqrt{\frac{\hat p (1 - \hat p)}{n}},$$ where $\hat p = x/n$ is the sample proportion, and $z^*_{\alpha/2}$ is the $\alpha/2$ upper quantile of the standard normal distribution. In your case $\alpha = 0.01$, so $z^*_{\alpha/2} \approx 2.32635$.

Note here that the standard error of the sample proportion is computed from the sample, because the confidence interval is a statistic. If we are to perform the corresponding hypothesis test, we must also use the corresponding variance estimate: that is to say, if $$H_0 : p = p_0 \quad \text{vs.} \quad H_1 : p \ne p_0$$ for a two sided test, where in your case the proportion under the null hypothesis is $p_0 = 0.72$, then the test statistic is $$Z \mid H_0 = \frac{\hat p - p_0}{\sqrt{\hat p (1 - \hat p)/n}} \sim \operatorname{Normal}(0,1),$$ and not $$Z \mid H_0 = \frac{\hat p - p_0}{\sqrt{p_0 (1 - p_0)/n}}.$$ The second one uses the null variance. To be clear, there are two versions of the one-sample proportion test; only the one that uses the standard error calculated from the sample will match the confidence interval.