1
vote
1answer
80 views

How is the entropy of the multivariate normal distribution with mean 0 calculated?

Here is what I have so far: $$\begin{align} h(x) &= - \int \frac{1}{(2\pi)^{\frac{D}{2}}\det\Sigma^{\frac{1}{2}}} \exp(-\frac{1}{2} x^T\Sigma^{-1}x) \ln ...
1
vote
1answer
75 views

About the differential entropies of well-known continuous distributions

Assume that the continuous random variable $X$ has a distribution (in a closed form expression) with differential entropy $h(X)$. Q) Then, is it true for any continuous distribution that the ...
0
votes
1answer
582 views

equivalence between uniform and normal distribution

The principle of insufficient reason says that all outcomes are equiprobable when we have no knowledge to guess otherwise. I understand this and that this corresponds to uniform distribution. However, ...
2
votes
1answer
54 views

Parameter optimization in probabilistic models

Task: Suppose we model a variable $y = Wx + \mu$ as a linear transformation of $x$ plus some Gaussian noise $\mu\sim\mathcal N(0,\sigma I)$. Our aim is to minimize the estimation error of $x$ given ...
1
vote
0answers
118 views

Incrementally compute the conditional entropy

Is it possible to compute a conditional entropy (see the two following formulas) in an incremental manner ? That is, the sets C and K are not fix: each time we have a new element c, set K may increase ...