We have a new observation $x_0$, whose response will be $Y_0 = \beta_0+\beta_1x_0+\epsilon_0$. We want to predict $Y_0$.

The estimator that we use is $\hat{Y}_0 = \hat{\beta}_0+\hat{\beta}_1x_0$.

The books goes on finding $E[\hat{Y}_0-Y_0]=0$ and $var[\hat{Y}_0-Y_0]=\sigma^2(1+h_{00})$. Also it says that since $Y_0$ is a random variable and it is normally distributed, we know that we can write $$0.95 = P\left(-c_1 < \frac{\hat{Y}_0 - Y_0}{\hat{\sigma} (1+h_{00})} < c_2 \right)$$ and then goes on finding the prediction interval.

So a couple of my questions are:

  • How do we know $Y_0$ is normally distributed? But more importantly, how do we know it is a random variable?
  • if $Y_0$ is a random variable, then why the responses $Y_i = \beta_0+\beta_1x_i+\epsilon_i$ are not treated as random variables? (or are they?)
  • Finally, if $Y_0$ and $Y_i$ are treated in the same way, i.e. they have the same distribution, how come their estimators $\hat{Y}_0 \equiv \hat{\beta}_0+\hat{\beta_1}x_0$ and $\hat{Y}_i \equiv \hat{\beta}_0+\hat{\beta_1}x_i$ are not the same thing?

They're all random variables. In simple linear regression each response, including the future response, is a normally distributed random variable because of the $\epsilon$ term, which by assumption has a normal distribution with zero mean.

When you derive formulas for statistical inference (confidence intervals, prediction intervals, etc.) you are manipulating random variables. The statement "let $X_1, X_2,\ldots, X_n$ be a random sample..." is an assertion that the $X$'s are random variables. This is the reason why you consult the normal table or the $t$ table or whatever table when finding critical values for confidence intervals -- you manipulate those random variables into a "pivotal" form (involving the random sample and the parameter of interest) that has that particular distribution.

The approach behind constructing confidence intervals for a parameter is to make a probability statement on this pivot, say $P(-t_{\alpha/2} < T < t_{\alpha/2}) = 1-\alpha$. You then rearrange the inequality into a equivalent statement -- a confidence interval -- with the parameter at the center, for which the probability statement still holds. Once you observe your random sample, you plug in the observed values into your formula, and presto! you've got the observed value for your confidence interval. The same logic applies to constructing prediction intervals for a future observation.

To answer your last question, the difference between $\hat Y_0$ and $\hat Y_i$ is that the future observation hasn't been observed yet. The beta hats involve only the observed variables, since the only prediction you can make on a future observation is one based on the responses you've observed.

  • $\begingroup$ Thank you first of all. Secondly, as you say $\epsilon_i \sim N(0, \sigma^2)$, i.e. $\epsilon_i$'s are normally distributed random variables. However, take for example $y_1$. We actually have the data and hence we actually have an observed value of $\epsilon_1$, which in this case won't be a random variable anymore, but rather a constant. Or is it still a RV? $\endgroup$ – Euler_Salter Apr 5 '17 at 16:02
  • $\begingroup$ Furthermore, is it correct to say that since $Y_i = \beta_0+\beta_1x_i+\epsilon_i$ then $E[Y_i] = \beta_0+\beta_1x_i$ and $Var[Y_i] = \sigma^2$? And if this is true, then is it correct to say that $Y_i \sim N(\beta_0+\beta_1x_i, \sigma^2)$? $\endgroup$ – Euler_Salter Apr 5 '17 at 16:05
  • $\begingroup$ @Euler_Salter You're right, the observed value of a RV is constant. When we derive a confidence interval the probability statements are referring to the random variables in their 'not yet observed' form. Once we plug in the observed values, the probability statement goes away. This is the reason why we have to say 'blah is a 95% confidence interval for $\mu$' rather than 'the probability is 95% that $\mu$ is between here and there'. $\endgroup$ – grand_chat Apr 5 '17 at 16:07
  • $\begingroup$ @Euler_Salter .. it's the same reason why $P(X=3)=1/6$ when $X$ is a die roll, but you can't assign probability $1/6$ to the observed die roll when you observe $x=3$. $\endgroup$ – grand_chat Apr 5 '17 at 16:09
  • $\begingroup$ @Euler_Salter Correct on your second question $\endgroup$ – grand_chat Apr 5 '17 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.