Suppose a fit a line using the method of least squares to $n$ points, all the standard statistical assumptions hold, and I want to estimate that line at a new point, $x_0$. Denoting that value by $\hat{y_0}$, the estimate is given by: $$\hat{y_0} = \hat{\beta_0} + \hat{\beta_1}x_0$$

Where $$\hat{\beta_0} = \bar{y} - \hat{\beta_1}\bar{x}$$ and $$\hat{\beta_1} = \frac{\sum_{i=1}^n (x_i-\bar{x})(y_i - \bar{y})}{\sum_{i=1}^n (x_i - \bar{x})^2}$$

I now want an expression for the variance of $\hat{y_0}$. I tried to do it as follows:

$$V(\hat{y_0}) = V(\hat{\beta_0}) + V(\hat{\beta_1}x_0) + 2Cov(\hat{\beta_0},\hat\beta_1x_0)$$

Where $V(\hat{\beta_0}) = 0$ since $\beta_0$ is constant by definition (depending on the sample means). I also think that the covariance term should be zero because $E(\hat{\beta_0}\hat{\beta_1x_0}) = E(\hat\beta_0)(\hat\beta_1x_0)$ intuitively, but I haven't proven this. I'm not sure what to do about the middle term -- do I just hammer out the computation?

Advice/Solution would be appreciated. Note: this is a review question for a test I have tomorrow, not homework. I need to understand this.


Actually, $\hat{\beta}_0$ isn't constant - it depends on the $y_i$ (which follow a normal distribution). Note that in most cases, this variance would be likely computed for a prediction interval since you're working with a new $x$-value.

Using properties of variances and covariances, $$\begin{align} \newcommand{\Var}[1]{\text{Var}\left(#1\right)}\newcommand{\Cov}[2]{\text{Cov}\left(#1, #2\right)}\Var{y_0} &= \Var{\hat{\beta}_0}+\Var{\hat{\beta}_1x_0} + 2\Cov{\hat{\beta}_0}{\hat{\beta}_1x_0} \\ &= \Var{\hat{\beta}_0}+x_0^2\Var{\hat{\beta}_1}+2x_0\Cov{\hat{\beta}_0}{\hat{\beta}_1} \end{align}$$ Now recall that the $y_i$ (not the predicted values) are based on $$y_i = \beta_0+\beta_1x_i+\epsilon_i\text{, } \epsilon_i \overset{iid}{\sim} \mathcal{N}(0, \sigma^2)$$

You can derive $\Var{\hat{\beta}_0}$ using the above, as here. You can also see here the derivation of $\Var{\hat{\beta}_1}$.

I haven't been able to find a derivation of the covariance. One way you could do this is by using $$\Cov{\hat{\beta}_0}{\hat{\beta}_1} = \Cov{\bar{y}-\hat{\beta}_1\bar{x}}{\hat{\beta}_1} = \Cov{\bar{y}}{\hat{\beta}_1} - \bar{x}\Var{\hat{\beta}_1}$$ and this might be helpful.

The derivation is very, very tedious and long, so I wouldn't expect to see this on an exam.

Unfortunately, it's been a long time since I've done these derivations, and I'm used to doing them using matrices (which, quite frankly, is a lot cleaner).

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.