# Noise Covariance Estimation for Linear Regression (Seemingly Unrelated Regressions)

Considering following linear model $$\begin{equation} y_t = X_t f_t + \varepsilon_t, \qquad t=1,\cdots, T \end{equation}$$ where $$y_t\in\Re^{300\times 1}$$ and $$X_t\in\Re^{300\times 60}$$ are two given sequences for $$t=1,\cdots, T$$. Error term $$\varepsilon_t\sim \mathcal{N}(0,Q)$$ and $$Q$$ is a $$300\times 300$$ unknown diagnal matrix.

I was wondering, is it possible to efficiently estimate the weights $$W = Q^{-1}$$ or the noise covariance of residuals? so that I could minimise the variance of the $$\hat{f}_t$$ by solving following weighted linear least squares: $$\begin{equation} \min_{f_t}\|y_t - X_t f_t\|^2_W \end{equation}$$

• I use Feasible generalized least squares to estimate the diagnal noise covariance iteratively: Feasible generalized least squares. It quickly converged within 50 iterations Jan 6 at 2:38
• Is there any structure on Q? If completely unrestricted, I don’t see how you’d estimate it. A typical strategy is to model Q as a parametric function of the regressors. Jan 10 at 5:15