# Tag Info

3

It is a probability density function. In particular, $p(x_0)$ is the probability of event $x_0$ occurring.

3

So the first arrow is simply using the basic identity of logarithms: $\log(a\cdot b) = \log{a} + \log{b}$. $$-\sum_{x \in \mathbb{X}}\sum_{y \in \mathbb{Y}} p(x,y) \log(p(x)p(y|x)) = -\sum_{x \in \mathbb{X}}\sum_{y \in \mathbb{Y}} p(x,y) \log{p(x) -\sum_{x \in \mathbb{X}}\sum_{y \in \mathbb{Y}} p(x,y)\log{p(y|x)}}$$ And the second arrow is just using the ...

1

A parenthesis might have helped. What he seemingly did was $$\log ( p(x) \cdot p(y|x) ) = \log (p(x)) + \log (p(y|x)),$$ splitting the sum into two parts. As for the second arrow he did no changes to the second term of the expression but apparently the first term is independent of $y$ and so he discarded the sum.

0

The notation being used is the following: For any $0\leq p\leq 1$, \begin{align} H(p,1-p)&= p\log\left(\frac{1}{p}\right)+(1-p)\log\left(\frac{1}{(1-p)}\right)\\ &=-\left(p\log(p)+(1-p)\log(1-p)\right) \end{align} represents the binary entropy function, and not the joint entropy of $X$ and $Y$. The convention is that $0\log(0)=0$. In ...

0

First, I believe $P_{expected}=(\frac{1}{6})^2$ because it doesn't matter what the first die is, just that the other 2 match. The value you give for $P_{expected}$ is the value for a specific value, say, 6. There are six such values, so $(\frac{1}{6})^3•6=(\frac{1}{6})^2$.Second, there is a difference between theoretical probability and experimental ...

3

A function is a mapping between a set of numbers and another set of numbers. A functional is a mapping between a set of functions and another set of functions. The entropy is defined as the Gibbs functional: $$S(p)=-k\sum_jp_j\log(p_j)$$ where the $p_j$ are functions. So the correct way to define the entropy, following Gibbs, is a functional

Top 50 recent answers are included