Tag Info

New answers tagged

0

Consider the following: \begin{align} h(X+Z)-h(Z)+h(Z)&=I(X;X+Z)+h(Z)\\ &=H(X)-H(X|X+Z)+h(Z)\\ & \le H(X)+h(Z)\\ &\le \log(A)+\frac{1}{2}\log(2 \pi eN) \end{align} Now combining with the bound you had \begin{align} h(X+Z) \le \min \left(\log(A)+\frac{1}{2}\log(2 \pi e N), \frac{1}{2}\log(2 \pi e(A^2+N) \right)\\ =\log(A)+\frac{1}{2}\log(2 ...


1

Just some hints. We can consider this as an additive gaussian channel, with noise $Z$, bounded input $X$ and output $W=X+Z$. Furthermore, $h(W) = h(Z) + h(X) - h(X|W)=h(Z)+I(X;W)$ For the first equality see here. Because $h(Z)$ is fixed, our problem of maximizing $h(W)$ is then equivalent to finding the pdf for $X$ that maximizes the mutual information, ...


0

Not sure I understand your exact question, but there is an explicit formula for the measure theoretic entropy of a first order Markov chain (that is what your directed, weighted graph represents). You need to compute the stationary vector $p$ for your stochastic matrix $P$ (the one given by the weights on your directed graph) and then the formula for the ...


2

Differential entropy I don't think that you can derive an upper bound for the following reason. Let's assume $Y \sim \mathcal N(0,\eta I)$ is Gaussian distributed (with $\eta$ chosen such that $H[Y]=n$) and $X \sim \mathcal N(\lambda O, \kappa I)$, where $\kappa$ is chosen such that $H[X]=n$, $O$ a matrix consisting of ones, and $\lambda$ a free parameter ...


1

This paper provides an example of a distribution on $\Bbb N$ which has infinite entropy. Thus, in your case $P$ is any partition satisfying $$ m(I_i) = \frac{1}{\lg(i+1)} - \frac{1}{\lg(i+2)}, \qquad i\in \Bbb N, $$ where $\lg i$ is a logarithm base $2$. For example, $I_1 = [1,\frac1{\lg3})$, $I_2 = [\frac1{\lg 3},\frac12)$, $I_3 = [\frac12,\frac1{\lg5})$ ...



Top 50 recent answers are included