# Tag Info

5

Let $x=\frac{p}{q}$ and $y=\frac{q^{\prime}}{p^{\prime}}=\frac{1-q}{1-p}.\;\;\;$ We know that $x>y$ since $\ln x>\ln y$, and we want to show that $\color{blue}{(p+q)\ln x>(p^{\prime}+q^{\prime})\ln y}$. Since $p=xq$ and $q^{\prime}=p^{\prime}y,\;\;$ $p+q=q(x+1)$ and $p^{\prime}+q^{\prime}=p^{\prime}(y+1)$. Then $\displaystyle ... 4 Let we assume$x,y\in(0,1)$and$|x-y|=h$. We want to prove: $$\sup_{\substack{x,y\in (0,1)\\ |x-y|=h}}\left|x \log x-y\log y\right|\leq -h\log h \tag{1}$$ but since$f(x)=x\log x$is a convex function on$I=(0,1)$, it is enough to prove$(1)$for$\min(x,y)\to 0^+$and$\max(x,y)\to 1^-$. The first case is trivial, since$\lim_{x\to 0^+}x\log x = 0$. So ... 2 I'm not sure if this counts, but it's an idea you might be interested in. In real world applications, you can identify a string in the$\Omega$alphabet that would never ever ever arise in a real message. Neither in the$\Omega$alphabet, nor in the$\Sigma$alphabet after the simple embedding from$\Omega$to$\Sigma$. Then you can use that string to ... 2 This is the entropy of a mixture of (two) distributions, the probability of sampling from distribution number$i$being proportional to the the sum$W_i$of the weights of the elements of multiset$i$(for$i=1,2$). The entropy is$\sum f(p) $where$f(p)=-p\log p$and$p$runs through the probabilities of the different elements, so$\sum f(tp_1 + ...

1

This is probably not what you are looking into (this a math site, not a programming site), but anyway: a simple conventional byte stuffing algorithm with some input shifting (each time we need to "escape", we change the offset, so as to miminize the probability of worst-case scenarios). Overhead (average) should be below 1%. Tested. Encoder: ...

1

We have $P(X=i)=\binom n i p^i (1-p)^{n-i}$. So $$H(X) = -\sum_{i=0}^n P(X=i)\lg P(X=i)$$ $$= -\sum_{i=0}^n \binom n i p^i (1-p)^{n-i}\left(\lg \binom n i+i\lg p+(n-i)\lg(n-i)\right).$$ Now let's look at each term: $$-\sum_{i=0}^n \binom n i p^i (1-p)^{n-i}\lg \binom n i=-E\lg\binom nX,$$ $$-\sum_{i=0}^n \binom n i p^i (1-p)^{n-i} i\lg p=-\lg p EX=-np\lg ... 1 Answering your questions on the minimisation: We can argue from the structure of the problem that the alphabet set of Z must be (a, a+1, a+2), for some integer a (Ask me about this if this is unclear). Set a = 0 for convenience, since the decoder will know a, and can always subtract it. Now, given this noise, we need to find the distribution ... 1 The author is trying to describe one of the distinctive qualities of Markov chains. In many stochastic processes, the value of the next symbol is entirely independent of the previous one. If, for example, we have x_n where x_n \in \{1, 2, 3, 4, 5, 6\} (eg outcomes when a die is thrown), then the value of x_{n+1} is independent of x_n. In a Markov ... 1 Not an answer at all, but a too-long-for-comment hint to those who are trying : The statement is equivalent to the following. For any positive numbers a_1,a_2,b_1,b_2 such that a_1+a_2=1 and b_1+b_2=1$$ \log(a_1/b_1) + \log(a_2/b_2) > 0\implies (a_1+b_1) \log(a_1/b_1) + (a_2+b_2) \log(a_2/b_2) > 0 This assertion seems to be true, but the ...

1

Having come back to this question and thought about it a bit more, I believe I have finally worked out how to formally express the sense of "invariance" that applies to Jeffreys' priors, as well as the logical issue that prevented me from seeing it before. The following lecture notes were helpful in coming to this conclusion, as they contain an explanation ...

1

The statement follows by observing that $X$ in you case is an average value. If you assume (I guess you did that) that the sequence is independent and identically distributed, then \begin{align} X&=-\frac{1}{n}\log P(x^n)\\ &=-\frac{1}{n}\log \left( \prod_{i=1}^n p(x_i)\right)\\ &= -\frac{1}{n}\sum_{i=1}^n \log p(x_i) \end{align} which is the ...

Only top voted, non community-wiki answers of a minimum length are eligible