0
$\begingroup$

My question is about confidence intervals of the slopes estimated in a multivariate regression.

I would like to clear up something that is probably a fault somewhere in my understanding. As I understand things:

1) $95\%$ confidence interval gives the region, around an estimate of y, for which we are $95\%$ confident that the true y lies, for a given x.

--> the narrower this interval, the more confident we are that the estimate lies closer to the true values - i.e. the better our estimate.

2) a higher t-stat (and therefore lower p-value) for a coefficient means we are more confident with the estimated value of that coefficient.

3) The width of the confidence interval is directly proportional the t-stat, (equation 3.2). A higher t-stat will produce a wider confidence interval.

3 means that 1 & 2 contradict each other. The two figs below show examples of how the confidence interval looks for a given t-stat, for two different explanatory variables of the same regression.

Fig 1: variable with low t-stat, high p-val, but 'narrow' confidence band Fig 1: variable with low t-stat, high p-val, but 'narrow' confidence band

Fig 2: variable with high t-stat, low p-val, but 'wide' confidence band Fig 2: variable with high t-stat, low p-val, but 'wide' confidence band

Can anyone clear this up? What intuition can we apply that explains why estimates with higher confidence (t-stat/p-value) have a wider confidence interval? Or am I misunderstanding?

Thanks, Chris

$\endgroup$
0
$\begingroup$

You're talking about two different concepts, although related. In particular, in the second case we may not have the $95\%$ confidence interval, so we are comparing different things.

  1. The width of the $95\%$ confidence interval for the t-Student distribution, is proportional to the standard deviation of the sample ($s$) and is smaller with larger sample sizes ($n$). For large $n$ it decreases roughly with $\sqrt{n}$. We choose the value of $t$ such that we are $95\%$ confident that the sample mean would be inside that interval whenever the null hypothesis is true, and $5\%$ confident that the sample mean will be outside of that interval.

  2. When we compute the value of $t$ and afterwards the $p$ value from the available data, we are in the background, sort of computing a confidence interval for some confidence, such that the sample mean is on the boundary of that interval whenever the null hypothesis is true. Therefore, we are not necessarily getting here the $95\%$ confidence interval but instead an interval with confidence $q = 1-p$. This means that if we created confidence intervals up to confidence $q$, the sample mean would be outside, assuming that the null hypothesis is true. Since we are observing some specific value of the sample mean, we can say that if the null hypothesis is true, we would have a confidence of $p$ of finding the sample mean at that distance or further away from the population mean. If the $p$ value is lower than some cut point, usually $5\%$, we would not be confident enough that the null hypothesis is true and so we would reject it.

So, in this second case, a higher $t$ value and, thus, a lower $p$ value mean that we would need to work with a larger confidence (not necessarily $95\%$) in order to not reject the null hypothesis.

$\endgroup$
  • $\begingroup$ (Part 1/2) Hi Ertxiem, thanks for your reply. I understand what you are saying about the P-value being used to reject the null hypothesis, but I don't follow why the confidence interval has to change for low p-values. To confirm, are you talking about the confidence interval for a prediction of y, from the regression coefficients? Or a confidence interval to reject the null hypothesis (and confirm confidence in the regression coefficient seen) based on the p-values? $\endgroup$ – Chris Browne Apr 29 at 10:57
  • $\begingroup$ (Part 2/2) My main confusion is why does the confidence interval for the prediction of y increase (and thus our confidence in our prediction of y decrease) with an increasing confidence in the estimate of the regression coefficient? Surely the confidence interval should get narrower with an increase in our confidence in the estimate? $\endgroup$ – Chris Browne Apr 29 at 10:59
  • $\begingroup$ The confidence interval being narrower for a fixed confidence is a good thing, because if means that we are being more precise in our estimate. However, if we want to be more confident that the "true" value is inside the confidence interval, we have to make it larger (therefore, a higher confidence is related with a smaller precision). $\endgroup$ – Ertxiem Apr 29 at 11:06
  • $\begingroup$ Hi Ertxiem, Agreed, I think we're saying the same thing? As you said, "the confidence interval being narrower for a fixed confidence is a good thing". If you look at the two graphs, these are both showing the 95% confidence interval either side of the regression line. The better estimate has a wider confidence interval, which doesn't seem to follow that logic. Any ideas why that is? $\endgroup$ – Chris Browne Apr 29 at 15:44
  • $\begingroup$ In this case it's tricky to compare the confidence intervals because the slopes are different in the two figures. In a regression, larger slopes result in wider confidence intervals. $\endgroup$ – Ertxiem Apr 29 at 23:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.