Tag Info

New answers tagged

0

Your intuition is good. Entropy is defined as the expected information gain, that is, how much you expect to learn from this event. If we were just looking at the first part, where you're given a random integer, as you have written, we would expect to learn $\log n$ bits of information. In the second part, with probability $\frac{1}{4}$ we will learn ...


1

We can first create a solution for small $t$ by simply looking at the behavior of the initial condition. Notice that at at $x=-1$ we have a shock, as we are jumping from $1$ to $0$, a rarefaction at $x=0$ as we go from $0$ to $2$, and finally a shock at $x=1$ as we go from $2$ to $0$. At $x=-1$, the R-H condition gives $\sigma = \frac{1}{2}$, and at $x=1$, ...


1

For one thing, $H(p)$ is not invertible, since $H(p)=H(1-p)$. Both $H(0)=H(1)=0$, so what would define $H^{-1}(0)$ to be? In theory this can be resolved by choosing a single branch of $H^{-1}$; you have two choices for each $H^{-1}$, so choose the greater one, for example. But finding the formula is still difficult. This is likely a transcendental ...


0

The first part of your question is a little confusing. $p_i$ cannot be an integer except 0 or 1. It's a probability and thus it's restricted to $0\leq p_i\leq 1$. The probabilities it represents are discrete. Regarding the other part of your question, the entropy is completely independent of the values that the function takes (the "alphabet" in information ...


0

Hint: I can tell you how it is obtained and you can work out. First you will simply write down the definition of entropy, which is the expected value of the logarithm of $1/P(X)$ where $X$ is your random variable which has a Binomial distribution. Then the one who found that result used Central Limit theorem. Using the CLT approximation of Binomial ...


0

Never heard of Greenberg-Hastings-Model, so I just googled it and according to Wikipedia this is a certain class of cellular automata (i.e. functions) on the full three shift over $\mathbb Z^2$ or a more general lattice. I am pretty sure the book you mention does not contain anything about this class of CA. I would instead recommend looking at books about ...


0

they are probabilities and when summed by $\sum_{i,j}$, cancel each other.


0

The entropy rate of a Markov chain with transition matrix $P$ and stationary distribution $\pi$ is $h(Q)=-\sum\limits_{i,j}\pi_iP_{i,j}\log P_{i,j}$. The specific transition matrix $P$ given in the question yields a Markov chain circling, after a while, deterministically on the states $4\to5\to6\to4$, hence $\pi_i=0$ for every $i$ in $\{0,1,2,3\}$ and ...


0

There seems to be a typo in your initial stochastic matrix though: Looking at your adjacency matrix A and the communicating classes you determined, I guess the entry $2/3$ should be in the same row, but one position to the right. Your algorithm to compute the TOPOLOGICAL entropy of your Markov chain (subshift of finite type) is correct. You could even take ...


1

$$ I(X;Y|Z) = \sum_{x,y,z}p(x,y,z)\log{p(x,y|z)\over p(x|z)p(y|z)} $$ so it isn't either of those things, but the mutual information between $X|Z$ and $Y|Z$ is a better description. [Reference: equation (2.118) of Information Theory and Network Coding by R. Yeung.]



Top 50 recent answers are included