3
$\begingroup$

Can someone explain this to me? I've read the relevant section of the text about a million times, and it was even explained in class, but I can't seem to wrap my head around it.

The Statement of the Problem:

Can $\sigma^2\{\text{pred}\}$ be brought increasingly close to $0$ as $n$ becomes large? Is this also the case for $\sigma^2\{\hat Y_h\}$? What is the implication of this difference?

Here

$$ \hat Y_h = \text{the point estimator of }E\{Y_h\}= b_0 + b_1X_h $$

$$ \text{and} $$

$$\sigma^2\{\hat Y_h\}=\text{the variability of the sampling distribution of }\hat Y_h=\sigma^2\left[ \frac{1}{n}+\frac{(X_h-\bar X )^2}{\sum (X_i - \bar X)^2} \right] $$

$$\text{and} $$

$$\sigma^2\{\text{pred}\} = \text{the variance of the prediction error}\\ =\sigma^2\{Y_{h(new)}-\hat Y_h\}=\sigma^2\{Y_{h(new)}\}+\sigma^2\{\hat Y_h\}=\sigma^2+\sigma^2\{\hat Y_h\}.$$

I know why, mathematically, it is the case that, as $n$ becomes large, $\sigma^2\{\hat Y_h\}$ approaches $0$, and $\sigma^2\{\text{pred}\}$ does not (it approaches $1$ instead, I believe). However, I'm looking for a more intuitive, conceptual understanding here. If someone could explain a little, or even just point me towards an explanation elsewhere on the web, I'd appreciate it. Thanks.

Also, if any other notation is unclear, let me know.

$\endgroup$

1 Answer 1

2
$\begingroup$

The estimate $\hat Y_k$ estimates the height of the regression line at the value $X_k$ of the predictor variable. Its variance is given your second displayed equation.

The predicted value $\hat Y_p = b_0 + b_1X_p$ corresponds to a new value $x_p,$ which is not part of the dataset used to find the estimated regression coefficients $b_0$ and $b_1.$ Its variance is is larger, as shown in your last displayed equation.

Sometimes this expression is given as

$$\sigma^2\{pred\} = V(\hat Y_p) = \sigma^2\left[1 + \frac{1}{n} + \frac{(X_p - \bar X)^2}{\sum_{i=1}^n (X_k - \bar X)^2} \right].$$

Roughly speaking, each of the three terms inside the large brackets has an intuitive meaning.

(a) The term $1/n$ expresses the component of error due to improperly estimating the $y$-intercept $\beta_0$ of the model as $b_0.$ This kind of error can be reduced by increasing the sample size.

(b) The term $\frac{(X_p - \bar X)^2}{\sum_{i=1}^n (X_k - \bar X)^2}$ expresses the component of error due to improperly estimating the slope $\beta_1$ of the model as $b_1$. The numerator disappears if the new predictor value is at the center of the previous data. (The estimated regression line 'pivots' at $(\bar X, \bar Y)$, the center of gravity of the data cloud.) The denominator tends to be larger with larger $n$; it is the numerator of the sample variance of the $X_i$. The slope is easiest to estimate if the $X_i$ are relatively widely scattered. (One of the few instances in statistics in which large variability is an advantage!)

(c) Finally, the term $1$ expresses the variability inherent in dealing with an $additional$ observation not previously used in estimating the regression line. The number of observations $n$ used to get the estimates $b_0$ and $b_1$ is not relevant to this component of error. (The overall variability $\sigma^2$ of the regression model is relevant because we assume the new point comes from the same bivariate population as for the previous $n$.)

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .