0
$\begingroup$

Show that the expected value of the fitted value at $\mathbf{x}_0$ is $\operatorname{E}(\hat{y}_0)=\mathbf{x}'_0\beta$ and its variance is $\operatorname{Var}(\hat{y}_0)=\sigma^2\mathbf{x}'_0(\mathbf{X'X})^{-1}\mathbf{x}_0$

This is a question about multiple linear regression. I am trying to figure out where to start.

$\endgroup$
1
  • $\begingroup$ The fitted value at $\mathbf{x_0'}$ is $\mathbf{x_0'} \widehat\beta.$ If you already know that $\operatorname E\widehat\beta=\beta$ and that $\mathbf{x_0}$ is constant, i.e. is not random, and that $\operatorname E\left( \mathbf{x_0'}\widehat\beta \right) = \mathbf{x_0'}\operatorname E\widehat\beta$ because $\mathbf{x_0'}$ is constant, then you've got the first part. Do you have some difficulty with some of that? $\endgroup$ Mar 9, 2020 at 2:56

1 Answer 1

0
$\begingroup$

You have $$ \mathbf X = \left[ \begin{array}{cccc} 1 & x_{11} & \cdots & x_{p1} \\ \vdots & \vdots & & \vdots \\ {} \\ {} \\ {} \\ {} \\ \vdots & \vdots & & \vdots \\ 1 & x_{1n} & \cdots & x_{pn} \end{array} \right] \in\mathbb R^{n\times(p+1)}. $$ This is a tall skinny matrix. And \begin{align} \widehat \beta & = \big(\mathbf{X'X}\big)^{-1} \mathbf{X'} Y \\[10pt] & = \underbrace{ \big(\mathbf{X'X}\big)^{-1} }_{(p+1)\times(p+1)} \,\, \underbrace{\,\,\, \mathbf{X'} Y\,\,\, }_{(p+1)\times 1} \\[10pt] \text{And so } & \widehat y_0 = \mathbf{x_0'} \widehat\beta \text{ is a scalar-valued random variable, and} \\[10pt] \operatorname{var} \left( \widehat\beta \right) & = \operatorname{var}\left( \mathbf{x_0'} \big( \mathbf{X'X} \big)^{-1} \mathbf{X'} Y \right) \\[10pt] & = \mathbf{x_0'} \big( \mathbf{X'X} \big)^{-1} \mathbf{X'} \Big( \operatorname{var}(Y) \Big) \mathbf{X} \big(\mathbf{X'X}\big)^{-1} \mathbf{x_0} \tag 1 \end{align} Here we applied the rules that $$ \operatorname{var}\big( \mathbf{A} Y \big) = \mathbf{A}\Big( \operatorname{var} Y \Big) \mathbf{A'} $$ and that $\mathbf{(AB)'} = \mathbf{B'} \mathbf{A'},$ i.e. the order of multiplication is reversed.

Now in line $(1)$, recall that $\operatorname{var}(Y) = \sigma^2 I_{n\times n}$ and do some routine cancelations.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .