Tagged Questions
0
votes
0answers
29 views
convexity of the product of two entropy-like functions
Consider the functions $T_p(q)= \sum_i q_i^p$, where p>1 and q is a finite-dimensional vector satisfying $\sum_i q_i = 1, q_i >0$ (ie, a probability mass function). In information-theoretic terms, ...
1
vote
0answers
75 views
Why is the negative entropy Lipschitz with respect to the $1$-norm (Over)?
Let $\left\|x \right\| = \sum_{i=1}^{i=n}\left|x^i\right|$ and $d\left(x\right)=\sum_{i=1}^{i=n}x^i\ln x^i$ where $x\in R^n $ and $ \sum_{i=1}^{i=n}x^i=1$
How to prove:
For all $x, x'$, $$\left| ...
1
vote
0answers
158 views
Is conditional entropy a convex function?
A conditional entropy can be expressed in the following way,
$H_{V_t}(V_s) = -\sum_{s,t}p(s,t)\log{p_t(s)} = -\sum_{s,t}p(s,t)\log{\frac{p(s,t)}{\sum_{s'}{p(s',t)}}}$
$s$ and $t$ are defined ...
1
vote
1answer
80 views
Upper bound for $-t \log t$
While reading Csiszár & Körner's "Information Theory: Coding Theorems for Discrete Memoryless Systems", I came across the following argument:
Since $f(t) \triangleq -t\log t$ is concave and ...
