We have a source $X$ with alphabet size equal to $N$. The Shannon entropy is defined as $$E(X)=-\sum _{i=1}^{N}p_{i}\cdot\log _{2}p_{i}$$ where $p_i$ is the probability of symbol $i$ appearing in the stream of characters of the message.

If the source transmits $M>N$ symbols so that the resulting signal is periodic (i.e. it "completes a pattern within a measurable time frame, called a period and repeats that pattern over identical subsequent periods", from Wikibooks) what is the entropy worth in this case?

  • $\begingroup$ $N$ is the alphabet size (how many available symbols there are to choose from at each time), or is the block length (related to the period?) $\endgroup$ – leonbloy Jan 29 at 15:31
  • $\begingroup$ @leonbloy thanks, I edited my post $\endgroup$ – Mark Jan 29 at 16:46
  • 1
    $\begingroup$ It should be $H(X)$ instead of $E(X)$. Also, I don't understand what relevance has $M>N$, and what relation (if any) has $M$ with the period. $\endgroup$ – leonbloy Jan 29 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.