I am trying to understand the weighted least squares estimation method, and I'd really appreciate it if you could shed some light on me. Let me explain my problem briefly:

Consider a linear model in a matrix form as $y=\beta x +e$ with $e \sim \mathcal{N} (0, \sigma^2I)$. To find an estimate of $x$, the weighted linear least squares estimator gives $$ \hat{x} = (\beta^tW\beta)^{-1} \beta^tW y, $$ where $W$ is the weight matrix with $w_{ii} = \sigma_i^{-2}$.

Assume that $\beta$ is known (and fixed). How sensitive is the WLS estimator ($\hat{x}$) with respect to the distortions of $y$? What is the relationship between the entries of $\hat{x}$ and $y$? Are there any relationships for the changes of $y$ such that $\hat{x}$ keeps the same values?

  • $\begingroup$ $\hat x$ is linear $y$... $\endgroup$ – user251257 Jan 7 '17 at 3:37
  • $\begingroup$ @user251257, Your comment is truncated. I am looking for a relationship. $\endgroup$ – Jolfaei Jan 7 '17 at 4:13
  • 1
    $\begingroup$ it is not truncated. The dependence is clearly given by your formula. Sensitivity of matrix multiplication is given by the so called condition number of the matrix. And of course if you add some nonzero vector from the null space to $y$, you won't change $\hat x$. There is nothing specific to WLS. Your question is only about matrix-vector-multiplication. The thing is different if you ask for the sensitivity in $\beta$ or $W$. $\endgroup$ – user251257 Jan 7 '17 at 4:40
  • 2
    $\begingroup$ If the weight is $\sigma^{-2}$ rather than $\sigma_i^{-2}$, i.e. if the weight does not depend on the index $i$, then this isn't really "weighted". $\qquad$ $\endgroup$ – Michael Hardy Jan 7 '17 at 5:25
  • $\begingroup$ @MichaelHardy thank you for pointing this out. You are right. It is $\sigma_i^{-2}$. I corrected it. $\endgroup$ – Jolfaei Jan 7 '17 at 22:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.