# What is the maximum entropy distribution over all integers (ie. including negative ones) with fixed mean and variance?

I know that the maximum entropy distribution with over the non-negative integers fixed mean is a geometric distributions.

However, I cannot find conclusive information about what are the maximum entropy distributions over the whole integers (negative + non-negatives).

As per the comment below, in this case, we need at least to specify both the mean and the standard deviation of the distribution for the "maximum entropy" criterion to make sense.

It is well known that if we consider the whole real line, the maximum entropy distribution with a given mean and variance is a Gaussian normal distribution.

My questions are therefore: 1- For a given mean and variance, is there a maximum entropy distribution over the integers? 2- If it exists, can it be expressed in a closed form, like gaussian or geometric distributions?

My guess is that the answer is "yes" for 1) and "no" for 2) (since I cannot find any mention of a closed form anywhere). But I would be very happy to see a confirmation of this.

EDIT: Since it is now clear that the variance needs to be specified for the "maximum entropy" criterion to make sense, I rewrote the question accordingly.

• There is no maximum entropy distribution in this case. You can always increase the entropy by spreading out the probability more, as long as you do it symmetrically about the mean. – Daniel Weissman Apr 1 '15 at 22:43
• Thank you. I was actually more or less expecting that. But then if we specify the variance as well, there should be a maximum entropy distribution, right? (like for the normal distribution in the real case). And if so, do we have a closed form for it? – Fabien C. Apr 2 '15 at 3:49

Solving the Lagrange Equations, we get that the maximum entropy distribution with mean $0$ and variance $1$ is where $$\sum_{k\in\mathbb{Z}}(k^2-1)e^{-ak^2}=0$$ which is $a\doteq0.4999998943842821\sim\frac12$. We need to compute the coefficient where $$c\sum_{k\in\mathbb{Z}}e^{-ak^2}=1$$ which is $c\doteq0.3989422361322933\sim0.3989422804014327=\frac1{\sqrt{2\pi}}$.
Thus, the maximum entropy distribution on the integers that has a mean of $0$ and variance of $1$, is $$p_k=c\,e^{-ak^2}$$ where $a$ and $c$ are given above. These values are extremely close to the Gaussian, which has the maximum entropy for a continuous distribution with the same constraints.
Although the function derived above is very close to the Gaussian distribution restricted to $\mathbb{Z}$, $\frac1{\sqrt{2\pi}}e^{-n^2/2}$ is not a probability measure on $\mathbb{Z}$. In fact, the Poisson Summation Formula says that \begin{align} \frac1{\sqrt{2\pi}}\sum_{n\in\mathbb{Z}}e^{-n^2/2} &=1+2\sum_{n=1}^\infty e^{-2\pi^2n^2}\\ &\gt1 \end{align}
• The standard Gaussian function, $\frac1{\sqrt{2\pi}}e^{-x^2/2}$, is not even a probability measure on $\mathbb{Z}$ because $$\frac1{\sqrt{2\pi}}\sum_{k\in\mathbb{Z}}e^{-k^2/2}=1.000000005350576$$ so the reference you cite cannot be correct. The correct answer, given above, is very close, but not exactly the standard Gaussian function on $\mathbb{Z}$. – robjohn Aug 3 '15 at 14:20
• Regarding the computation of $a$ and $c$, I simply used numerical methods to solve $$\sum_{k\in\mathbb{Z}}(k^2-1)e^{-ak^2}=0$$ for $a$. Once we have $a$, it is a simple division to find the $c$ so that $$c\sum_{k\in\mathbb{Z}}e^{-ak^2}=1$$ – robjohn Aug 3 '15 at 14:39