Suppose there are i.i.d. binary random variables $X_i \sim X$ with distribution $P(X=1) = 0.75$ and $P(X=0) = 0.25$

i) For $n=5$ and $e=0.1$, which sequences fall in the typical set $A_e^n$? What is the probability of $A_e^n$?

ii) How many elements are in the essential bit content set $S_e$ for $X^5$ for $e = 0.1$?

  • $\begingroup$ Welcome to MSE! It really helps to format questions using MathJax (see FAQ). Also, what have you tried? What are your thoughts on the problem? Regards $\endgroup$ – Amzoti Apr 8 '13 at 4:21
  • $\begingroup$ i cant use mathjax because the school network does not permit the installation of software. I tried listing "00000", "00001", "00010",.... "11111" for A(e)^n but it is very exhaustive. I think that e is the error but am not sure how to find the probability. A(x) is the ensemble of {0,1} $\endgroup$ – Ice Apr 8 '13 at 4:30
  • $\begingroup$ MathJax is a markup language that you use when you edit your question. You do not need to install anything. What is $A$? $\endgroup$ – copper.hat Apr 8 '13 at 5:19
  • $\begingroup$ May I know how to use MathJax in edit? I am clueless about using MathJax in text, I can only see the links, attachment, headers, etc option. 'A' is the ensemble. $\endgroup$ – Ice Apr 8 '13 at 14:25
  • $\begingroup$ I've edited the question with formatting. Please take a look at it (click "edit") to see how it works (math formatting correspond to content between \$ \$ ) $\endgroup$ – leonbloy Apr 8 '13 at 18:36

First, compute the entropy:
$H = - \frac{1}{4} \log \frac{1}{4} - \frac{3}{4} \log \frac{3}{4} = 0.561 $ (bits/symbol)

So, $2^{-n H(X)} = 0.1431$

Now, the probability of a given 5-sequence with $k$ ones is $0.75^{k} \times 0.25^{5-k}$ Compute and tabulate this for each $k$, and see which sequences fall in the respective $e-$typical set.

Is the above clear for you? Can you go on from here?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.