There are many algorithms (like Huffman, Arithmetic) which exploit the redundancy in the source message stream and compress the source symbols before sending it over (noisy/noiseless) channel to the receiver.

The information theory resources assume that the source symbols are known (given). Then, they explain various algorithms which take advantage of the uneven symbol probabilities in the source message to construct the final code.

There are two steps for encoding a source message

  1. Identify source symbols (mostly a set of ascii characters)
  2. Construct a code using source symbol probabilities (Huffman, Arithmetic etc)

My question is regarding step 1.

How can we identify the set of source symbols?. For instance one can simply use ascii set as source symbols, but can we do better than this? Is there any algorithm to identify the source symbol (not simply ascii characters) and then encode so that overall code has minimum length? Just as variable length codes, can we have variable length symbols?

  • $\begingroup$ What do you mean identify? We are talking about a digital channel, aren't we? $\endgroup$ – Hagen von Eitzen Oct 12 '15 at 6:48
  • $\begingroup$ Identify a set of symbols that can be encoded using Huffman etc. For instance one can simply take an ascii set as source symbols, can we do better than this? $\endgroup$ – Curious Oct 12 '15 at 6:51
  • 1
    $\begingroup$ You could switch from single symbols (ASCII letters) to compounds (digrams, syllables, words, phrases), which may increase compressibility but also requires you to transmit the "dictionary" ... $\endgroup$ – Hagen von Eitzen Oct 12 '15 at 18:33
  • $\begingroup$ is there a way (algorithm) to decide which symbols (digrams or syllables or words or phrases or ascii) are more appropriate? Is it possible to use variable length symbols? (dictionary can contain unigram, bigram as well as trigram) $\endgroup$ – Curious Oct 13 '15 at 7:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.