# A question on confidence

So, I've been reviewing some of my old stats courses in preparation for an interview I have in a couple of days. I'm a bit stuck on a particular question and hope you could help.

A drug trial gives the result that the drug works better than the placebo, with 95% conﬁdence. What exactly does this statement mean? What further assumptions are needed to be able to deduce that the probability of the drug working is actually 95%?

My answer to the first part is... 95% confidence means that there is a 1 in 20 chance that the difference could have been observed by chance i.e. if the experiment was conducted many times.

Any suggestion for part 2?

-

95% confidence means that 95% of the time, your test will produce intervals that contain the true mean.

In your case, it means that the actual success of the drug is higher than the actual success of the placebo 95% of the time (from a large number of tests).

The assumptions are: the sample size is adequately large, there do not exist biases in how the test was conducted, etc.

-
Do confidence levels and intervals have meaning only when the results are normally distributed and the sample size is large etc or can they be used in non-normal small-sample situations as well? –  Dilip Sarwate Jun 5 '12 at 16:40
Confidence intervals have the same meaning in context when the distribution is non-normal and the sample size is small. Yet, they are calculated differently than a confidence interval for a normal distribution. It might be difficult to attain a 95% confidence interval with those conditions though (as an increased sample size increases confidence levels). Look at the derivation for more information en.wikipedia.org/wiki/Confidence_interval#Theoretical_example –  rckrd Jun 5 '12 at 16:56

First part: the probability that the trial result or something better would be seen by chance (i.e. the proportion if the trial was repeated a very large number of times) , if the drug is in fact just as good as a placebo, is less than or equal to 5%.

Second part: A key assumption (or estimate from past experience) is an a priori figure for how often drugs in fact work better than placebos.

-