# Covariance Matrix in Weighted Least Square Estimation

I am new to linear algebra and I have the following doubts:

1. In weighted least square estimation of the system $Ax = b$ we minimize the weighted value of the error $e = b - Ax$ and the best $\hat{x}$ is given by $( A^T \Sigma^{-1}A )^{-1} A^T\Sigma^{-1} b$ where $\Sigma$ is the covariane matrix of the error $e$. Why is the covariance matrix $\Sigma{e}$ the best choice for the weighting matrix? Is there any derivation for it? Please refer me to its link or sum hints will also do.

2. For the same linear system $e = b - Ax$ is $E(ee^T) = E(bb^T)$ given that error is unbiased (i.e. $E(e) = 0$)?

-

First, I think we need to clarify the problem statement. The system here should be $y=Ax+v$ where $y$ is the measurement (I prefer to use $y$ instead of $b$), and $v$ is a zero-mean random noise who is uncorrelated to $x$. In this system, $x$ is deterministic and $v$ and $y$ are stochastic.
Second, the estimator $\hat{x}=Ky$ with $K=(A^T\Sigma^{-1}A)^{-1}A^T\Sigma^{-1}$ is a linear minimum variance unbiased estimator of $x$. Note $\mathbb{E}(v)=0$ it is easy to check $\mathbb{E}(\hat{x})=x$ and $\mathbb{E}{(x-\hat{x})(x-\hat{x})^T}=(A^T\Sigma^{-1}A)^{-1}$. The proof that the estimation variance is the minimum can be found on page 97 of "Linear estimation".
Third, I think $\mathbb{E}(ee^T)=K\Sigma K^T$ instead of $\mathbb{E}(ee^T)=\Sigma$. Hint: $KA=I$ because $\mathbb{E}\hat{x}=KAx=x$.