Consider n independent observations ${(x_i,y_i) : 1 ≤ i ≤ n}$ from the model

$Y = α + βx + \epsilon$,

where $\epsilon$ is normal with mean $0$ and variance $σ^2$. Let $\hat \alpha,\hat β \text{ and } \hat {\sigma}^2 \text{ be the maximum likelihood estimators of } α , β \text{ and } {\sigma}^2$, respectively. Let $v_{11}, v_{22} \text{and } v_{12}$ be the estimated values of $Var(\hat α), Var(\hat β) and Cov(\hat α,\hat β)$, respectively.

(a) What is the estimated mean of Y when $ x = x_0$? Estimate the mean squared error of this estimator.

(b) What is the predicted value of Y when $x = x_0$? Estimate the mean squared error of this predictor.

Here is what i know :-

I think by estimated mean of Y they want $E[Y|X=x_0]$. (I may be wrong!!!) then the estimator shall be $ \alpha + \beta x_0$ . Now I donot know how to proceed about finding the MSE.

b) the preicted value of Y , when $x=x_0$ is $\hat \alpha + \hat β x_0 + \epsilon $ . Then the MSE shall be equal to $Var (\hat \alpha + \hat β x_0 +\epsilon) +{bias (\hat \alpha + \hat β x_0 + \epsilon)}^2$ , which turns out to be $\rightarrow v_{11} + x_{0}v_{22} + 2x_{0}v_{12}$ , Since bias = 0.

Any corrections /help is appreciated.Thanks


Let us write down the covariance matrix of the estimated coefficients \begin{align} \Sigma = \begin{pmatrix} v_{11} & v_{12} \\ v_{21} & v_{22} \end{pmatrix}, \end{align} where $v_{12} = v_{21}$.

The estimated mean is $\hat{y}(x_0) = \hat{\alpha} + \hat{\beta} x_0$, its variance is given by

\begin{align} \operatorname{var}(\hat{y}(x_0))&= \operatorname{var}( (1, x_0) \Sigma_{\hat{\beta}})\\ &= (1, x_0) \Sigma_{\hat{\beta}} (1, x_0)^T\\ &= \operatorname{var}(\hat{\alpha})+x_0^2\operatorname{var}(\hat{\beta})+ 2x_0 \operatorname{cov}(\hat{\alpha},\hat{\beta})\\ &=v_{11} + x_0^2v_{22}+2x_0v_{12}. \end{align} Recall that in this case $\hat{\alpha}$ and $\hat{\beta}$ are the random variables and $x_0$ is the constant.

While for the prediction, just add up the variance of the noise term (that is independent of the estimators), i.e.,

$$ \operatorname{var}(\hat{y}(x_0) + \epsilon)= \operatorname{var}(\hat{\alpha})+x_0^2\operatorname{var}(\hat{\beta})+ 2x_0 \operatorname{cov}(\hat{\alpha},\hat{\beta})+ \sigma^2 . $$

  • $\begingroup$ I found a duplicate question on SSE but the answers are quite different .Hope you could clarify a little bit. stats.stackexchange.com/questions/95546/… $\endgroup$ – DRPR Mar 28 '18 at 8:59
  • $\begingroup$ They are not different. In $Var( aX + bY)$ the $X$ and $Y$ are the r.v.s, in our case the estimated coefficients are the r.v.s while $X=x_0$ is constant. $\endgroup$ – V. Vancak Mar 28 '18 at 11:48
  • $\begingroup$ In a general sense, what is the difference between an estimated mean and a predicted value (except for the inclusion of \epsilon ) $\endgroup$ – DRPR Mar 28 '18 at 15:41
  • $\begingroup$ Well. There is no difference in the point estimator. The OLS/MLE procedure estimates the expected value of $Y$ given $X=x$. Hence, when we have to find confidence intervals for the prediction itself it is not enough to account for the sample uncertainty in the coeff. estimation, as the value $Y_i$ comes from $\mathbb{E}[Y|X]$ plus the noise $\epsilon$, hence we have to add the (estimated) variance of the noise. $\endgroup$ – V. Vancak Mar 28 '18 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.