# variance of multiple regression coefficients

If I consider universal kriging (or multiple spatial regression) in matrix form as:

${\bf{V = XA + R }}$

where $\bf{R}$ is the residual and $\bf{A}$ are the trend coefficients, then the estimate of ${\bf{\hat A}}$ is:

${\bf{\hat A}}=(\bf{X^{T}C^{-1}X)^{-1}X^{T}C^{-1}V}$

(as I understand it), where $\bf{C}$ is the covariance matrix, if it is known. Then, the variance of the coefficients is:

$\text{VAR}({\bf{\hat A}})=(\bf{X^{T}C^{-1}X)^{-1}}$???

I am getting this from here.

How does one get from the estimate of ${\bf{\hat A}}$, to its variance? i.e. how can I derive that variance?

-
Maybe I'm something of a purist on certain issues. See this article: en.wikipedia.org/wiki/Errors_and_residuals_in_statistics What you're talking about here seems to be an error rather than a residual. – Michael Hardy Jun 28 '12 at 21:43
Yes, you are probably right. I'm coming from a geostatistics background where we model a random variable (i.e. rock porosity) often by separating the data into a trend component and a residual component (I have seen it referred to as both error and residual). The trend component is modelled deterministically and the residual is modelled either deterministically or stochastically. – Flux Capacitor Jun 28 '12 at 22:32

$$\newcommand{\var}{\operatorname{var}}$$ First, recall that $$\var(MV) = M\Big(\var(V)\Big)M^T.$$ so \begin{align} & \var((X^T C^{-1}X)^{-1} X^T C^{-1}V) \\[10pt] & = (X^T C^{-1}X)^{-1} X^T C^{-1}\Big(\var{V}\Big)\Big( (X^T C^{-1}X)^{-1} X^T C^{-1} \Big)^T. \tag{1} \end{align}
Then, recall that $(AB)^T$ (with $A$ to the left of $B$) is equal to $B^T A^T$ (with $A$ to the right of $B$). With $X^T C^{-1} X$, one cannot invert all three matrices and multiply in the opposite order, since $X$ is not a square matrix. But that matrix is symmetric, i.e. it is its own transpose. And $C$ is also symmetric, and so is $C^{-1}$. So we get: $$\Big( (X^T C^{-1}X)^{-1} X^T C^{-1} \Big)^T = C^{-1}X(X^TC^{-1}X)^{-1}.$$ Then $(1)$ becomes \begin{align} & (X^T C^{-1}X)^{-1} X^T C^{-1}\Big(\var{V}\Big) C^{-1}X(X^TC^{-1}X)^{-1} \\[10pt] & = (X^T C^{-1}X)^{-1} X^T C^{-1}\Big( C \Big) C^{-1}X(X^TC^{-1}X)^{-1} \\[10pt] & = (X^T C^{-1}X)^{-1} X^T C^{-1} X(X^TC^{-1}X)^{-1} \\[10pt] & = (X^T C^{-1}X)^{-1}. \end{align}