# Inverse of a sub-matrix

I have a multivariate Gaussian distribution with known $\mu$ and $\Sigma$. I want to evaluate it given a vector $x$. However, some of the attributes of this vector may be unknown, in which case I want to marginalize those values out.

This effectively reduces it to another MV Gaussian distribution with parameters $\hat{\mu}$ and $\hat{\Sigma}$ that are obtained after dropping the rows (and columns) corresponding to unknown attributes in $x$. (Please correct me if I am wrong).

For efficiency of implementation, I pre-compute the inverse of $\Sigma$ and determinant ($|\Sigma|$) to be used while evaluating the distribution. However, when $x$ has unknown values, is there a simple way of determining the inverse of $\hat{\Sigma}$ from the pre-computed inverse of $\Sigma$? Similarly, can I find the determinant easily for $\hat{\Sigma}$?

-
If unsuccessful here, try stats.stackexchange.com –  user53153 Feb 12 '13 at 22:11
I was about to ask an identical question. Any suggestions appreciated. –  conjectures Feb 21 '14 at 8:53

Partition your covariance matrix $\Sigma$ and its inverse $\Psi$ as follows, with the subscripts $_A$ denoting known values and $_B$ denoting missing values:

$$\Sigma = \begin{pmatrix}\Sigma_A & \Sigma_{AB} \\ \Sigma_{BA} & \Sigma_B\end{pmatrix}, \qquad\Psi = \begin{pmatrix}\Psi_A & \Psi_{AB} \\ \Psi_{BA} & \Psi_B\end{pmatrix}.$$

If we then invert $\Psi$, we find:

$$\Sigma_A = (\Psi_A - \Psi_{AB}\Psi_B^{-1}\Psi_{BA})^{-1},$$

i.e.

$$\Sigma_A^{-1} = \Psi_A - \Psi_{AB}\Psi_B^{-1}\Psi_{BA}$$

If the number of missing values is small, it will be easier to invert $\Psi_B$ than to directly invert $\Sigma_A$.

Similar manipulations can get us from

$$\det(\Sigma) = \det(\Sigma_A)\det(\Sigma_B-\Sigma_{BA}\Sigma_B^{-1}\Sigma_{AB}),$$

to the required formula

$$\det(\Sigma_A) = \det(\Sigma)\det(\Psi_B).$$

Note that the partitioning doesn't have to be contiguous, since we could have included permutation matrices which would have cancelled out in the resulting formulae.

-