Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Consider a linear model $$ y = Ab+n, $$ where $b \in \mathbb{R^m}$ is a parameter to be estimated, $n \in \mathbb{R^{n}}$ is a noise with mean $\mathbb{E}n = m_{n}$ and with covariation matrix $R_{n}$. The Gauss-Markov estimator is an umbiased estimator of parameter $b$ given by $\beta(y) = B(y-m_{n})$, where $B$ is a solution of an optimisation problem $$ \mathbb{E}(b - B(y-m_{n}))^2 \to \min\limits_{B}. $$ Solving this problem we receive $$ \beta(y) = (A'R_{n}^{-1}A)^{-1}A'R_{n}^{-1}(y-m_{n}). $$ I'm looking for properties of this estimator. Would it have the minimum variance in the class of unbiased linear estimators? Would it be consistent? And if the answer is negative in general case, maybe it is true if $n$ is a gaussian vector?

share|cite|improve this question
all good, except it need not be consistent. You can't consistently fit a line if all but one of your observations are at one point – mike Apr 22 '12 at 12:27
And what about the minimal variance? – Nimza Apr 22 '12 at 12:34
I think so. And I would think the usual complete sufficient stat argument would do it. – mike Apr 24 '12 at 18:54
up vote 1 down vote accepted

Yes, if $\tilde{\beta} = Hy$ is a different unbiased linear estimator of $b$, then the difference $\textrm{COV}(\tilde{\beta}) - \textrm{COV}(\beta)$ is a positive definite matrix.

Therefore, the variance of any unbiased linear estimate of $b$ (any element of the vector) which is not given by $(A'R_{n}^{-1}A)^{-1}A'R_{n}^{-1}(y-m_{n})$ is always higher.

The proof is quite simple and follows the Wikipedia entry for the Gauss Markov estimator:

The estimator is unbiased:

$E(Hy) = E((A' R_n^{-1} A)^{-1} A' R^{-1}+D)(Ab+n)=\\ = (A' R_n^{-1} A)^{-1} H' R^{-1}+D)Ab= \\ =(A'R_n^{-1} A)^{-1} A' R_n^{-1} Ab+DAb= \\ =(I+DA)b$

Therefore, the product DA must be 0 (we assume $E(n)=0$)

The covariance of the estimate: $\textrm{COV} (Hy)=H \textrm{COV} (y) H'= [(A' R_n^{-1} A)^{-1} A' R_n^{-1}+D] R_n [(A' R_n^{-1} A)^{-1} A' R_n^{-1}+D]'= \\ =(A' R^{-1} A)^{-1}+DR_nD'$

Where the last equality is due to DA=0. The result follows since the last term is always positive definite if $D \neq 0$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.