$Q$ is a rectangular matrix with orthonormal columns. A linear system composed of $$Qx= b$$ is really easy to solve as:
$$Q'Q=I$$ hence: $$x=Q'b$$
Given that $Q$ is orthonormal can this be used to solve a weighted linear regression problem efficiently? The normal equation for this system is:
$W_i$ is a diagonal matrix with only positive entries along the ridge. For each different $W_i$ I do a Cholesky factorization of the normal equation with the associated cost $O(n^3)$. Are there methods that would do better.
For some back ground reading: Iteratively reweighted least squares