0
$\begingroup$

Suppose there are i.i.d. binary random variables $X_i \sim X$ with distribution $P(X=1) = 0.75$ and $P(X=0) = 0.25$

i) For $n=5$ and $e=0.1$, which sequences fall in the typical set $A_e^n$? What is the probability of $A_e^n$?

ii) How many elements are in the essential bit content set $S_e$ for $X^5$ for $e = 0.1$?

$\endgroup$
9
  • $\begingroup$ Welcome to MSE! It really helps to format questions using MathJax (see FAQ). Also, what have you tried? What are your thoughts on the problem? Regards $\endgroup$
    – Amzoti
    Apr 8, 2013 at 4:21
  • $\begingroup$ i cant use mathjax because the school network does not permit the installation of software. I tried listing "00000", "00001", "00010",.... "11111" for A(e)^n but it is very exhaustive. I think that e is the error but am not sure how to find the probability. A(x) is the ensemble of {0,1} $\endgroup$
    – Ice
    Apr 8, 2013 at 4:30
  • $\begingroup$ MathJax is a markup language that you use when you edit your question. You do not need to install anything. What is $A$? $\endgroup$
    – copper.hat
    Apr 8, 2013 at 5:19
  • $\begingroup$ May I know how to use MathJax in edit? I am clueless about using MathJax in text, I can only see the links, attachment, headers, etc option. 'A' is the ensemble. $\endgroup$
    – Ice
    Apr 8, 2013 at 14:25
  • $\begingroup$ I've edited the question with formatting. Please take a look at it (click "edit") to see how it works (math formatting correspond to content between \$ \$ ) $\endgroup$
    – leonbloy
    Apr 8, 2013 at 18:36

1 Answer 1

1
$\begingroup$

First, compute the entropy:
$H = - \frac{1}{4} \log \frac{1}{4} - \frac{3}{4} \log \frac{3}{4} = 0.561 $ (bits/symbol)

So, $2^{-n H(X)} = 0.1431$

Now, the probability of a given 5-sequence with $k$ ones is $0.75^{k} \times 0.25^{5-k}$ Compute and tabulate this for each $k$, and see which sequences fall in the respective $e-$typical set.

Is the above clear for you? Can you go on from here?

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .