0
$\begingroup$

I understand log of determinant of covariance matrix bounds entropy for gaussian distributed data. Is this the case for non gaussian data as well and if so, why?

What does Determinant of Covariance Matrix give? and http://web.ntpu.edu.tw/~phwang/teaching/2012s/IT/slides/chap08.pdf show connection between 'differential entropy' and log of determinant of covariance matrix for Gaussian case. Eqn. 26 of https://arxiv.org/pdf/1604.03924.pdf?fbclid=IwAR1tDOzgZ2iXSo3lDbXnr8TUkxawCA8NikHFlfY4E5OWmbmJ3_WHeVPotFE has some relations (I guess for non-Gaussian case, but yet to check that)

$\endgroup$
  • $\begingroup$ For context, please mention some of the bounds (names/statements) of the bounds that you have seen. This will help others attract your attention towards similar bounds for possibly non-Gaussian data. $\endgroup$ – астон вілла олоф мэллбэрг Feb 9 at 5:39
  • $\begingroup$ added the links! $\endgroup$ – hearse Feb 9 at 6:24
1
$\begingroup$

I understand log of determinant of covariance matrix bounds entropy for gaussian distributed data

Wrong. That's not a bound, it's the value of entropy for a gaussian. And it's not exactly log of determinant of covariance matrix but

$$H(Z)=\frac{k}{2} \ln(2 \pi e)+\frac12 \ln (| \Sigma|)$$

Is this the case for non gaussian data as well and if so, why?

It's a well known result that the gaussian distribution maximizes the entropy, for a given covariance. See eg here.

Hence, yes, that value is a bound.

$\endgroup$
  • $\begingroup$ What does 'previous value' refer to in your answer? For the non gaussian case it being a bound , is what you meant ? $\endgroup$ – hearse Feb 9 at 15:38
  • $\begingroup$ Yes, I meant the value of the equation above. $\endgroup$ – leonbloy Feb 9 at 16:15
  • $\begingroup$ is there a similar bound or any bound for that sake on cross-entropy based on determinants? $\endgroup$ – hearse May 2 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.