$Y=XB+u$ where $X$ is a non random $n\times k$ Matrix, $\textrm{rank}(X)=k, E(u)=0, E(uu')=\sigma^2\Omega$, How to form $(1)$ How to proof $(2)$ the general Gauss-Markov theorem?


closed as off-topic by StubbornAtom, Xander Henderson, Adrian Keister, dantopa, Lee David Chung Lin May 16 at 5:02

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question is missing context or other details: Please provide additional context, which ideally explains why the question is relevant to you and our community. Some forms of context include: background and motivation, relevant definitions, source, possible strategies, your current progress, why the question is interesting or important, etc." – StubbornAtom, Xander Henderson, Adrian Keister, dantopa, Lee David Chung Lin
If this question can be reworded to fit the rules in the help center, please edit the question.


The Gauss-Markov Theorem states that the OLS estimator:

$$\hat{\boldsymbol{\beta}}_{OLS} = (X'X)^{-1}X'Y$$

is Best Linear Unbiased. For the proof, I will focus on conditional expectations and variance: the results extend easily to non conditional. Also, for the proof, I consider $I_{n}$ $=$ $\Omega$, but the result extends easily to the non equal case as well.


Is it unbiased?

$$E(\hat{\boldsymbol{\beta}}_{OLS} \mid X) = E[(X'X)^{-1}X'Y \mid X] = E[(X'X)^{-1}X'(X\boldsymbol{\beta} + u) \mid X] = \\ \boldsymbol{\beta} + (X'X)^{-1}X'E(u \mid X) = \boldsymbol{\beta}$$

Yes! Is it, then, among the unbiased, that with the smallest variance? Consider, for this purpose, a general linear unbiased estimator $\boldsymbol{b}$:

$$\boldsymbol{b} = C\boldsymbol{y}$$

where $C$ is a generic $k$ $\times$ $n$ matrix that depends only on the sample information in $X$ and, given unbiasedness, such that $CX$ $=$ $I_{k}$ to guarantee unbiasedness. Note that for $\hat{\boldsymbol{\beta}}_{OLS}$, $C_{Ols}$ $=$ $(X'X)^{-1}X'$. It can be proved that:

$$Var(\hat{\boldsymbol{\beta}}_{OLS} \mid X) = \sigma^{2}(X'X)^{-1}$$

and, for the generic linear estimator:

$$Var(\boldsymbol{b} \mid X) = \sigma^{2}(C'C)^{-1}$$.

We can additionally define $D$ $=$ $C$ $-$ $C_{ols}$. It is immediate that $DX$ $=$ $0$. From it, we can finally conclude that:

\begin{align} Var(\boldsymbol{b} \mid X) &= \sigma^{2}[D - (X'X)^{-1}X'][D - X(X'X)^{-1}] \\ &= \sigma^{2}(X'X)^{-1} + \sigma^{2}DD' \\ &= Var(\hat{\boldsymbol{\beta}}_{OLS} \mid X) + \sigma^{2}DD' \\ &> Var(\hat{\boldsymbol{\beta}}_{OLS} \mid X) \end{align}

Since DD' is non negative defined.

  • $\begingroup$ Hello. We discourage giving full answers to questions with zero input from the asker. $\endgroup$ – StubbornAtom May 15 at 13:24
  • $\begingroup$ Thanks for the note, StubbornAtom: a form of netiquette I wasn't aware of. $\endgroup$ – Nicg May 15 at 13:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.