Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a multivariate Gaussian distribution with known $\mu$ and $\Sigma$. I want to evaluate it given a vector $x$. However, some of the attributes of this vector may be unknown, in which case I want to marginalize those values out.

This effectively reduces it to another MV Gaussian distribution with parameters $\hat{\mu}$ and $\hat{\Sigma}$ that are obtained after dropping the rows (and columns) corresponding to unknown attributes in $x$. (Please correct me if I am wrong).

For efficiency of implementation, I pre-compute the inverse of $\Sigma$ and determinant ($|\Sigma|$) to be used while evaluating the distribution. However, when $x$ has unknown values, is there a simple way of determining the inverse of $\hat{\Sigma}$ from the pre-computed inverse of $\Sigma$? Similarly, can I find the determinant easily for $\hat{\Sigma}$?

share|improve this question
    
If unsuccessful here, try stats.stackexchange.com –  user53153 Feb 12 '13 at 22:11
    
I was about to ask an identical question. Any suggestions appreciated. –  conjectures Feb 21 at 8:53

1 Answer 1

Partition your covariance matrix $\Sigma$ and its inverse $\Psi$ as follows, with the subscripts $_A$ denoting known values and $_B$ denoting missing values:

$$ \Sigma = \begin{pmatrix}\Sigma_A & \Sigma_{AB} \\ \Sigma_{BA} & \Sigma_B\end{pmatrix}, \qquad\Psi = \begin{pmatrix}\Psi_A & \Psi_{AB} \\ \Psi_{BA} & \Psi_B\end{pmatrix}. $$

If we then invert $\Psi$, we find:

$$ \Sigma_A = (\Psi_A - \Psi_{AB}\Psi_B^{-1}\Psi_{BA})^{-1}, $$

i.e.

$$ \Sigma_A^{-1} = \Psi_A - \Psi_{AB}\Psi_B^{-1}\Psi_{BA} $$

If the number of missing values is small, it will be easier to invert $\Psi_B$ than to directly invert $\Sigma_A$.

Similar manipulations can get us from

$$ \det(\Sigma) = \det(\Sigma_A)\det(\Sigma_B-\Sigma_{BA}\Sigma_B^{-1}\Sigma_{AB}), $$

to the required formula

$$ \det(\Sigma_A) = \det(\Sigma)\det(\Psi_B). $$

Note that the partitioning doesn't have to be contiguous, since we could have included permutation matrices which would have cancelled out in the resulting formulae.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.