# Tagged Questions

This tag is for [Entropy](http://en.wikipedia.org/wiki/Entropy_(information_theory)) in Mathematics.

35 views

44 views

### Is my entropy calculation correct? Clustering entropy example

I would like to calculate entropy of this example scheme http://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-clustering-1.html Equation of entropy Then the entropy is (the first line) ...
97 views

### Mutual information vs Information Gain

I always thought that mutual information and information gain refer to the same thing, however looking at Wikipedia: http://en.wikipedia.org/wiki/Information_gain https://en.wikipedia.org/wiki/...
13 views

35 views

### Relative entropy for wiener measure/wiener measure with girsanov change of drift

I've read an article on relative entropy properties that gives a result for the relative entropy of two equivalent measures as they are found in applications of girsanovs theorem. For two measures P, ...
29 views

### Comparison between Shannon's and Blackwell's measure of informativeness

I want to compare the concept of precision of information'' between signals $x \in X$ and states $\omega \in \Omega$ defined by Blackwell and Shannon. Denote the conditional probability ...
29 views

### Proof that the q-Gaussian is derived by maximizing Tsallis entropy

So I have to write a report paper for a course and I would like to prove that the q-Gaussian is the distribution which arises once one maximizes the Tsallis entropy. But I face difficulties in ...
55 views

### Is this some entropy I haven't heard of?

For a discrete finite probability distribution $p(s)$ the function $$\sum\limits_s p(s)\log ^2 p(s)$$ looks like the Shannon entropy but has a square on the $\log$. Is there a name for this? Or it is ...
162 views

### What is the entropy correction term for a multivariate distribution?

I wonder if anyone would be able to help me with a confusion I have got myself into please. Consider a fixed and given $n$ by $n$ matrix $M$ whose elements are chosen from $\{-1,1\}$. Consider also a ...
35 views

### Multivariate Gaussian with singular covariance matrix

The entropy of a multivariate Gaussian is given at https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Entropy as $$\frac{1}{2}\ln((2\pi e)^n |\Sigma|).$$ Here $n$ is the dimension of the ...
156 views

### Convergence to normal distribution

Consider the probability distribution of the simple symmetric walk. That is the random variable $X_i$ equals $c$ or $-c$ with equal probability and all $X_i$ are independent and $c\geq1$. We are ...
29 views

### Approximating the set of 2-typical sequences

I have been trying to find an upper bound for the set of 2-typical sequences; here is how far I got - I would appreciate any further help very much: Let $x^n=x_1,x_2,\ldots, x_n$ be a sequence from a ...
43 views

62 views

### Mutual information for a continuous uniform distribution

I'm trying to compute using matlab the mutual information for an $\infty$-PAM input (the limit of a very dense PAM constellation) for a range of snr and I got stuck. I'm working with a real-valued ...
22 views

### Lower bound on conditional entropy over multiple random variables

I am trying to compute the best subset of features for a given random variable $X_i$ from the set of given $n$ random variables. For that I am using conditional entropy to determine the best subset, ...
### Special functions related to $\sum _{n=1}^{\infty } \frac{x^n \log (n!)}{n!}$
While doing some caculation related to von Neumann entropy, I encountered this kind of convergent series. $$\text{Exl}(x) \equiv \sum _{n=1}^{\infty } \frac{x^n \log (n!)}{n!}$$ In my calculation, ...