1
$\begingroup$

Here is what my notes say about estimation and prediction:

Estimating the conditional mean

"We need to estimate the conditional mean $\beta_0+\beta_1x_0$ at a value $x_0$, so we use $\hat{Y_0}=\hat{\beta_0}+\hat{\beta_1}x_0$ as a natural estimator." here we get $$\hat{Y_0} \sim N\left(\beta_0+\beta_1x_0,\sigma^2h_{00}\right) \,\,\,\,\,\,\,\,\,\,\,\ \text{where} \,\,\,\,\,\,\,\,\,\,\,\ h_{00} = \frac{1}{n}+\frac{(x_0-\bar{x})^2}{(n-1)s_x^2}$$ with a confidence interval for $E(Y_0) =\beta_0+\beta_1x_0$ is $$\left(\hat{b_0}+\hat{b_1}x_0-cs\sqrt{h_{00}},\hat{b_0}+\hat{b_1}x_0+cs\sqrt{h_{00}}\right)$$ where $c = t_{n-2,1-\frac{\alpha}{2}}$ Where these results are found by looking at the shape of the distribution and at $E(\hat{Y_0})$ and $var(\hat{Y_0})$

Predicting observations

"We want to predict the observation $Y_0 = \beta_0+\beta_1x_0+\epsilon_0$ at a value $x_0$" $$E(\hat{Y_0}-Y_0) = 0 \,\,\,\,\,\,\,\,\,\,\ \text{and}\,\,\,\,\,\,\,\,\,\ var(\hat{Y_0}-Y_0) = \sigma^2(1+h_{00})$$ Hence a prediction interval is of the form $$\left(\hat{b_0}+\hat{b_1}x_0-cs\sqrt{h_{00}+1},\hat{b_0}+\hat{b_1}x_0+cs\sqrt{h_{00}+1}\right)$$

So I understand that in theory the prediction interval should be larger because of bigger uncertainty. I understand that we have this thanks to the $+1$ under square root. BUT MY QUESTION IS: What is the difference in meaning between the two? What is the difference in mathematical terms between the two? For example why on earth do we look at $E(\hat{Y_0}-Y_0)$ for the prediction interval? Of course we are going to get something different from before. Can you please give me an insight and intuition on why we do what we do and what we actually do?

$\endgroup$
6
  • $\begingroup$ and also, what is the prediction interval for? Like is it an interval for $x_0$ ? $\endgroup$
    – Euler_Salter
    Mar 5 '17 at 13:46
  • $\begingroup$ Possibly your model looks something like $Y=\beta_0+\beta_1x+\epsilon$, so your estimate of $Y_0$ is more uncertain than your estimate of $\beta_0+\beta_1x_0$ $\endgroup$
    – Henry
    Mar 5 '17 at 13:52
  • $\begingroup$ as my title says, my model is a simple linear regression model, so yes we have $Y_i = \beta_0+\beta_1x_i+\epsilon_i$. I don't understand your point though.. $\endgroup$
    – Euler_Salter
    Mar 5 '17 at 13:54
  • $\begingroup$ Estimating the conditional mean has uncertainty about the values of $\beta_0$ and $\beta_1$. Predicting $Y$ has that uncertainty and the uncertainty of a particular value of $\epsilon$ $\endgroup$
    – Henry
    Mar 5 '17 at 13:57
  • 1
    $\begingroup$ Estimation uses existing data to a CI for $E(Y)$ for a given value of $x.$ If you have a new value $x_0$ and want a CI for $E(\hat Y),$ where $\hat Y = \hat \beta_0 + \hat \beta_1 x_0,$ with $\hat \beta_0$ and $\hat \beta_1$ estimated by existing data, then there is extra variability. Suppose a study leading to approval of a new drug shows response $\hat Y = \hat \beta_0 + \hat \beta_1 x_0$ when subject was given dose $x.$ Now MD has new patient, meeting protocol of research subjects, and wants a pred interval for resp if he/she gives dose $x_0$ to new patient. That's example of practical use. $\endgroup$
    – BruceET
    Mar 5 '17 at 21:05
1
$\begingroup$

You have to distinguish between estimating $E[Y|X=x_0]$ and ${Y}(x_0)$. For the former you have that $$ \widehat{E[Y|X=x]}=\hat{\beta_0} + \hat{\beta}_1x_0, $$ such that $$ Var( \widehat{E[Y|X=x]}) = \sigma^2[1/n + (x_0 - \bar{x})^2/((n-1)s_x^2)] = \sigma^2 h_{00}, $$ where for the latter you have that $$ Var( \hat{Y}(x_0)) = Var( \widehat{E[Y|X=x]}) + Var(\epsilon_0)= \sigma^2(1+h_{00}). $$ Namely, for the former case you are estimating the conditional mean of $Y$ at $x_0$, while for the latter you are estimating (predicting) the value itself. The conditional mean "smooths out" the variance of the error term as $E[Y|X=x]= \beta_0 + \beta_1x$, however the prediction of a new value $Y(x_0)$ should consider also the (estimated) variance of the error term (to account for the fluctuation around the conditional mean).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.