Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Assume that the continuous random variable $X$ has a distribution (in a closed form expression) with differential entropy $h(X)$.

Q) Then, is it true for any continuous distribution that the log-standard deviation is additively separable from the entropy expression s.t. $$h(X) = \log \sigma + C,$$ where $\sigma$ is the standard deviation of distribution and $C$ may be a function of parameters of distribution? If my claim is false for some continuous distributions, then for what family of distributions exactly does the claim hold?

I emphasize that when $\sigma$ is dependent on a shape parameter of the distribution, the shape parameter must fixed so that varying $\sigma$ only varies the "spread'' of distribution. Let me illustrate with a few examples.

For a Gaussian random variable $X$, we see \begin{align} h(X)=\log(\sigma \sqrt{2 \pi e}) = \log \sigma + C, \end{align} where $C = \log \sqrt{2 \pi e}$.

For an Erlang random variable $X$, we see \begin{align} h(X) &= \frac{1}{\psi(k)}(1-k) + \log \frac{\Gamma(k)}{\lambda} + k, \\ &= \log \frac{\Gamma(k) \sigma}{\sqrt{k}}+ \frac{1}{\psi(k)}(1-k) + k \\ &= \log { \sigma}+ C, \end{align} where $k$ is the shape parameter and $C = \log \frac{\Gamma(k)}{\sqrt{k}}+ \frac{1}{\psi(k)}(1-k) + k $. Notice how I used the fact that $k$, a shape parameter, is fixed but $\lambda$ may be not when $\sigma$ is varied.

share|cite|improve this question
I do not understand completely. Do you mean to say that $h(X) = \log \sigma + C$,where $C$ shouldn't depend on $\sigma$? What about this – Kumara Feb 22 '13 at 5:24
I am sorry that I didn't include the condition but I am assuming that the variance of the distribution is finite. – tatterdemalion Feb 22 '13 at 13:22

If $\sigma$ only affects the "spread" of $X$, as you require, then you can write $X = \mu + \sigma Y$, where the distribution of $Y$ does not depend on $\mu$ or $\sigma$. In that case you have indeed $h(X) = h(\sigma Y) = h(Y) + \log \sigma$.

share|cite|improve this answer
What do you mean by spread? – Kumara Feb 25 '13 at 6:11

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.