Suppose we have a linear regression model of the following format : $$ y(x) = \beta_0 + \beta_1 x_1+ \beta_2x_2+\beta_3x_3+\epsilon$$

We know that the prediction interval associated with a level $\alpha$, for a new observation $x_0$, is :

$$ IC_{1-\alpha}=\left[\hat{y}_i \pm t_{\alpha/2, n-p-1}\times \sqrt{\hat{\sigma}^2(1+x_0^T(X^TX)^{-1}x_0)}\right]$$

The asked question is :
Without calculation and using approximations, give a prediction interval at $95\%$ for a given $(x_1,x_2,x_3)$.
Available information :

  • Output of the summary of this regression model given by R (so we have the $\beta s$, $\hat{\sigma}^2$ and $std(\beta)$)
  • The means and std errors of our variables.

We can approximate $t_{\alpha/2, n-p-1}$ by 2, how can we approximate $x_0^T(X^TX)^{-1}x_0$ ?


The covariance matrix of $\hat{\beta}$ you can obtain in R by vcov(model). If you are unwilling to do that then can crudely approximate $\hat{\sigma}^2(X'X)^{-1}$ by a diagonal matrix where its diagonal consists out of the variances of each $\hat{\beta}_j$, respectively. These values (standard deviation) you can find in the summary output.

  • $\begingroup$ As we don't have access to R for the question, the second answer is what I was looking for. Could you please elaborate on why the diagonal matrix of the variances of $\hat{\beta}$ is a good/decent approximate of $\hat{sigma}^2(X'X)^{-1}$ ? Thanks. $\endgroup$ – mlx Dec 17 '18 at 18:48
  • $\begingroup$ It depends whether your model is stable. If you have complete multicoliniearity, then $(X'X)$ is singular, on the other extreme if the $x$s are orthogonal $\sigma^2(X'X)$ is diagonal as the non-diagonal entries are the covariances of the $\hat{\beta}_s$. Hence, given that your model is pretty stable (low standard deviations of $\hat{\beta}$s and significant model) then you can consider neglecting this covariance terms. $\endgroup$ – V. Vancak Dec 17 '18 at 18:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.