Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

For mixture of multivariate Bernoulli distribution we have that,

$$p(x|\mu,\pi) =\Sigma_{k=1}^{K}\pi_kp(x|\mu_k)$$ where $$p(x|\mu_k) = \prod_{i=1}^{D}\mu_{ki}^{x_i}(1-\mu_{ki})^{1-x_i}$$

I read it from the book that

$$E[x] = \Sigma_{i=1}^{K}\pi_k\mu_k$$ $$cov[x] = \Sigma_{k=1}^{K}\pi_k(\Sigma_k+\mu_k\mu_k^T) - E[x]E[x]^T$$

The mean is trivial to prove, however I can't find proof for the covariance and I don't know how to prove it.

Can anyone help?

share|improve this question

1 Answer 1

up vote 1 down vote accepted

You know that, in general, $cov[x] = E[x x^T] - E[x]E[x]^T$ and you want to compute the first term. We can use the property: $E[g(X)] = E [ E(g(X)|k]]$

But $E(x x^T | k)$ (i.e., fixing the population 'k' index of the mixing), is $\Sigma_k+\mu_k\mu_k^T$ (don't let the $k$ subindexes confuse you: they are the first and seconds moments of $x$, for a fixed population index $k$).

Now, the above is a function ok $k$, and its expectation is $\sum_{k=1}^K \pi_k (\Sigma_k+\mu_k\mu_k^T)$

Notice that this result is not really connected to the Bernoulli distribution, it's valid in general.

share|improve this answer
    
Hi, can I know why $$E(xx^T|k) = \Sigma_k+\mu_k\mu_k^T$$ I think that's the thing that confuses me –  Jing Nov 30 '12 at 4:55
    
@Jing: It's just the same first equation of my answer, rearranged, and applied not to $x$ (the mixed variable) but to $x_k$ (one of the $K$ components of the mixture) –  leonbloy Nov 30 '12 at 11:50

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.