1
$\begingroup$

Suppose I have $n$ observations, which are all normally distributed with the same mean (which is unknown) but each has a different variance (the different variances could be called $v_1,...,v_n$ for example, which are all assumed to be known).

What is the best estimate of the mean? And further, how does one compute the variance of the mean?

Clearly, the sample average would be the best estimate if iid, but intuitively, an observation which has a relatively low variance would provide more information about the mean than the others, suggesting that the sample mean would not be the best estimate. My problem is that I don't know how to 'quantify' this intuition to generate an estimate of the mean and it's variance. Any ideas on how to do this? Thanks.

$\endgroup$
1
$\begingroup$

This is a very good question. It must surely be standard, but I don't know the relevant keywords, so failed to find an answer.

Anyway, it seems reasonable to consider the modified least squares estimator: $$ \hat \mu = \underset{\mu\in\mathbb R}{\operatorname{argmin}} \sum_{k=1}^n \frac{(X_k-\mu)^2}{v_k} = \frac{\sum_{k=1}^n v_k^{-1}X_k}{\sum_{k=1}^n v_k^{-1}}; $$ for example, this is an MLE for Gaussian observations.

$\endgroup$
  • $\begingroup$ While I was typing, @drhab's answer appeared. It seems (I'm too lazy to check) that it gives the same estimator. $\endgroup$ – zhoraster Sep 23 '17 at 14:03
  • $\begingroup$ I am too lazy too :-) $\endgroup$ – drhab Sep 23 '17 at 14:03
  • $\begingroup$ Thanks, both give the same answer... and above is consistent with some MC simulations I have done, and must be the best estimate. $\endgroup$ – Anna Efron Sep 23 '17 at 14:12
2
$\begingroup$

I preassume that the $X_i$ (i.e. the observations) are independent.

Let's take $X:=\lambda_1X_1+\cdots+\lambda_nX_n$ where $\lambda_1+\cdots+\lambda_n=1$.

Then $\mathbb EX$ equals the mean and: $$\text{Var}(X)=\lambda_1^2v_1+\cdots+\lambda_n^2v_n$$

To be found is a minimal value of this variance under the condition $\lambda_1+\cdots+\lambda_n=1$.

Lagrange multipliers can be handsome here.

$\endgroup$
  • 1
    $\begingroup$ So the weight goes as the reciprocal of the variance after suitable normalization. This is a standard result and is used in meta analysis when combining trials. $\endgroup$ – user121049 Sep 23 '17 at 16:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.