# Standard deviation and probability in a problem

I'm practicing for a probability midterm, and I was wondering if someone can tell me if my answers are right or not.

Arkansas recently passed a law sparking a debate about a new treatment. The treatment is supposed to help a person’s pregnancy continue under conditions that usually end it.

Usually, under those conditions, only about 40% of people’s pregnancies continue. But in a small case study, six pregnant people were given the new treatment, and four of them continued their pregnancies.

Critics say this result is not significant: it doesn’t show the treatment has an effect.

Use a normal approximation to assess this criticism. The null hypothesis is that each pregnancy is a “trial” with 0.4 probability of “success”.

(a) What are μ and σ in the case study?

In this case, the mean is 4 people. The standard deviation is $\sqrt(npq) = \sqrt(10*0.4*0.6) = 1.5$

(b) Are the results of the case study significant at the .05 level?

I believe this is a personal question, personally I'm assuming this means two standard deviations away or 3 people, which I'd say yes since 3 people out of 10 is a lot.

(c) Suppose a much larger study were done, with 150 subjects getting the treatment, and two/thirds of them continuing their pregnancies. What would μ and σ be then? Would the results be significant at the .01 level?

This makes μ $= 150 * 2/3 = 100$, and σ = $\sqrt(150*2/3*1/3) = 5.77$. In this case, .01 is is three standard deviations away, which means 17 people. Personally, that still seems worth it.

Speaking to this point, the hypothesis to be tested is $$H_0 : p = p_0, \quad \text{vs.} \quad H_a : p \ne p_0,$$ where $p$ is the true proportion of continuation of pregnancy in the treated population, and $p_0 = 0.4$ is the true proportion of continuation of pregnancy in the treatment-naive population. Thus, the purpose of the test is to determine if the sample contains sufficient evidence to suggest that the treatment has some effect on the proportion of continuation of pregnancy.
The sample you observed contains $X = 4$ such continuations ("responders to treatment") out of $n = 6$ observations. Thus the sample mean proportion of responses is $\hat p = X/n = 4/6 = 2/3$. This is your point estimate of the true proportion $p$ of responders to treatment, given the sample you observed. Under the assumption that the null hypothesis is true, and under the normal approximation, we have $$\hat p \mid H_0 \sim \operatorname{Normal}(\mu = p_0, \sigma^2 = p_0 (1-p_0)/n).$$ That is to say, $\hat p \mid H_0$ is approximately normal with mean $p_0 = 0.4$ and variance $p_0(1-p_0)/n = 0.04$. Then the test statistic $$(Z \mid H_0) = \frac{\hat p - p_0}{\sigma} = \frac{2/3 - 0.4}{0.2} = 1.3333$$ follows a standard normal distribution, and the statistic we calculated above gives us a $z$-score. The probability of observing such a $z$-score at least as extreme as what we calculated, is quite large: $$\Pr[|Z| > 1.3333] = 0.182422.$$ This is the $p$-value of the test. Since this $p$-value exceeds the significance level of $\alpha = 0.05$, we conclude that the given sample contains insufficient evidence to suggest that the treatment is associated with a change in the proportion of responders at the $5\%$ significance level.
The reason for this result is quite clear: the sample is much too tiny to be of any meaningful use. This is why the question proceeds to the second part, in which the sample size is $n = 150$, yet the point estimate remains $\hat p = 2/3$. In this case, the estimated standard error of the proportion is much, much smaller: $\sigma^2 = 0.0016$, meaning that the test statistic is now $$(Z \mid H_0) = \frac{2/3 - 0.4}{0.04} = 6.66667,$$ and here we clearly reject $H_0$ since $$\Pr[|Z| > 6.66667] = 2.61678 \times 10^{-11} \ll 0.05.$$ We would properly conclude that a sample of $n = 150$ subjects of which $100$ responded to treatment, would furnish sufficient evidence that the treatment changes the response rate at the $5\%$ significance level.