I am currently trying to figure out the parameter covariance for a weighted linear least squares problem where $$y = X\beta$$ The parameters for which my objective function is lowest are given by $$\hat{\beta} = (X^TWX)^{-1}X^TWy $$

I have a linear model, which has two sources of errors in my case. Each measurement is perturbed by process and measurement noise. The process error is constant and has a variance of $\sigma^2$. My measurements have variadic errors $σ^2_i$. So $\Sigma = diag(\sigma^2_i \dots \sigma^2_n)$. I have an estimate of $\Sigma$, but not of $\sigma^2$.

What I am interested in now is the sample of covariance $\hat{\beta}$ given $X$ and $y$. The Wikipedia article on weighted linear least squares uses error propagation to get to $$M^\beta = (X^TWX)^{-1}X^TW M W^TX(X^TWX)^{-1}$$ Where $M$ is the error covariance matrix of the samples. If I let $M=Is^2 + \Sigma$, how can I provide $s^2$? In the article this is based on the objective function at $\hat{\beta}$, which makes sense. I tried to test this on the case where $\beta$ is single variate, and I thought this should revert to the weighted sample variance, but it doesn't seem to do.

  • Is my assumption correct, that it should revert to the weighted sample variance in the single variate case?
  • What should my choice of $s^2$ be?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.