Geometric distribution achieves maximum entropy for given mean Let $X$ be a random variable with geometric distribution, ie $P(X=k)=p(1-p)^k$. If I calculated it correctly, $X$ has mean $E(X)=\frac{1-p}p$ and entropy $H(X)=-\log p - \frac{1-p}p\log{(1-p)}$ (edited because I want to start at $0$). Now I'd like to show there is no non-negative integer valued random variable Y with $E(Y)=E(X)$ and $H(Y)>H(X)$.
There is something on Wikipedia but I can't quite follow it. Is there an elementary way to show a maximum entropy distribution must be of the form $Cr^k$? How does the maximality of the geometric distributon follow?
/edit: Just realised it's not geometric but almost: $k$ instead of $k-1$ in the exponent.
 A: Take your geometric random variable distributed as $p_k=\mathbb P (X=k)=p(1-p)^{k}$ for $k\in\{0,1,2,\dots\}$. Note that your entropy is:
$$
H(X)=-\log p-\frac{1-p}{p}\log(1-p).
$$
And consider another random variable $Y$ distributed as $q_k=\mathbb P (Y=k)$ such that:
$$
\frac{1-p}{p}=\mathbb E(X)=\mathbb E(Y)=\sum_{i=0}^\infty i q_i.
$$
We use following lemma, which is nothing but positiveness of KL-divergence:
Lemma For the random variables $X$ and $Y$ distributed as $p_k\neq 0$ and $q_k$ on $k\in\{0,1,2,\dots\}$, we have
$$
\mathbb D(Y|X)=\sum_0^\infty q_i\log \frac{q_i}{p_i}\geq 0.
$$
The proof of lemma is really simple and it is omitted. Now use the lemma to see that :
$$
H(Y)=-\sum q_i\log q_i\leq -\sum q_i\log p_i\\
=-\sum q_i (\log p+i\log(1-p))=-\log p-\log(1-p)\mathbb E(Y)\\
=-\log p-\frac{1-p}{p}\log(1-p)=H(X).
$$
A: Not a solution to your question, but just to let you know that for a geometric random variable $X$,
$$P(X=k)=p(1-p)^{k-1}$$
From the definition of expectation or by looking at the moment generating function, you should have that
$$E(X)=\dfrac{1}{p}$$
The entropy $H(X)$ is then given by
$$H(X) = \sum_{k=1}^{\infty} -(1-p)^{k-1}p \cdot \log_{2}{((1-p)^{k-1}p)}$$
$$\quad =- \log_{2}{(p)} - \frac{(1-p)\log_{2}{(1-p)}}{p}$$
