Consider n independent observations ${(x_i,y_i) : 1 ≤ i ≤ n}$ from the model

$Y = α + βx + \epsilon$,

where $\epsilon$ is normal with mean $0$ and variance $σ^2$. Let $\hat \alpha,\hat β \text{ and } \hat {\sigma}^2 \text{ be the maximum likelihood estimators of } α , β \text{ and } {\sigma}^2$, respectively. Let $v_{11}, v_{22} \text{and } v_{12}$ be the estimated values of $Var(\hat α), Var(\hat β) and Cov(\hat α,\hat β)$, respectively.

(a) What is the estimated mean of Y when $ x = x_0$? Estimate the mean squared error of this estimator.

(b) What is the predicted value of Y when $x = x_0$? Estimate the mean squared error of this predictor.

Here is what i know :-

I think by estimated mean of Y they want $E[Y|X=x_0]$. (I may be wrong!!!) then the estimator shall be $ \alpha + \beta x_0$ . Now I donot know how to proceed about finding the MSE.

b) the preicted value of Y , when $x=x_0$ is $\hat \alpha + \hat β x_0 + \epsilon $ . Then the MSE shall be equal to $Var (\hat \alpha + \hat β x_0 +\epsilon) +{bias (\hat \alpha + \hat β x_0 + \epsilon)}^2$ , which turns out to be $\rightarrow v_{11} + x_{0}v_{22} + 2x_{0}v_{12}$ , Since bias = 0.

Any corrections /help is appreciated.Thanks


1 Answer 1


Let us write down the covariance matrix of the estimated coefficients \begin{align} \Sigma = \begin{pmatrix} v_{11} & v_{12} \\ v_{21} & v_{22} \end{pmatrix}, \end{align} where $v_{12} = v_{21}$.

The estimated mean is $\hat{y}(x_0) = \hat{\alpha} + \hat{\beta} x_0$, its variance is given by

\begin{align} \operatorname{var}(\hat{y}(x_0))&= \operatorname{var}( (1, x_0) \Sigma_{\hat{\beta}})\\ &= (1, x_0) \Sigma_{\hat{\beta}} (1, x_0)^T\\ &= \operatorname{var}(\hat{\alpha})+x_0^2\operatorname{var}(\hat{\beta})+ 2x_0 \operatorname{cov}(\hat{\alpha},\hat{\beta})\\ &=v_{11} + x_0^2v_{22}+2x_0v_{12}. \end{align} Recall that in this case $\hat{\alpha}$ and $\hat{\beta}$ are the random variables and $x_0$ is the constant.

While for the prediction, just add up the variance of the noise term (that is independent of the estimators), i.e.,

$$ \operatorname{var}(\hat{y}(x_0) + \epsilon)= \operatorname{var}(\hat{\alpha})+x_0^2\operatorname{var}(\hat{\beta})+ 2x_0 \operatorname{cov}(\hat{\alpha},\hat{\beta})+ \sigma^2 . $$

  • $\begingroup$ I found a duplicate question on SSE but the answers are quite different .Hope you could clarify a little bit. stats.stackexchange.com/questions/95546/… $\endgroup$
    – DRPR
    Commented Mar 28, 2018 at 8:59
  • $\begingroup$ They are not different. In $Var( aX + bY)$ the $X$ and $Y$ are the r.v.s, in our case the estimated coefficients are the r.v.s while $X=x_0$ is constant. $\endgroup$
    – V. Vancak
    Commented Mar 28, 2018 at 11:48
  • $\begingroup$ In a general sense, what is the difference between an estimated mean and a predicted value (except for the inclusion of \epsilon ) $\endgroup$
    – DRPR
    Commented Mar 28, 2018 at 15:41
  • $\begingroup$ Well. There is no difference in the point estimator. The OLS/MLE procedure estimates the expected value of $Y$ given $X=x$. Hence, when we have to find confidence intervals for the prediction itself it is not enough to account for the sample uncertainty in the coeff. estimation, as the value $Y_i$ comes from $\mathbb{E}[Y|X]$ plus the noise $\epsilon$, hence we have to add the (estimated) variance of the noise. $\endgroup$
    – V. Vancak
    Commented Mar 28, 2018 at 16:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .