2
$\begingroup$

I am unable to solve this homework question and was hoping to get some help for part (a).

I know that $Z$ is a stationary process because its generating function does not change with time. By the opposite reasoning, $W$ is not stationary because the coin flip happens at every time $n$.

For the second part of (a), I had thought that $Z$ would be IID because the probability of the events are independent and occur with the same distribution. $W$ is not IID because of the coin flip occurring at every time $n$ which means its distribution at time $n$ is different than all other times.

The answer turned out to be the opposite but I don't understand why.

enter image description here

PS: Don't worry its not an actual homework https://documents.epfl.ch/groups/i/ip/ipg/www/2015-2016/Information%20Theory%20and%20Coding/hw5.pdf

$\endgroup$
2
$\begingroup$

The process $W$ is indeed i.i.d., and $Z$ is not, unless the distribution of $\{X_n\}_{n=-\infty}^\infty$ is the same as the distribution of $\{Y_n\}_{n=-\infty}^\infty$.

Let us assume the distribution of $\{X_n\}_{n=-\infty}^\infty$ is not the same as the distribution of $\{Y_n\}_{n=-\infty}^\infty,$ so there is some set $A$ of values of those random variables for which $\Pr(X_n\in A)\ne\Pr(Y_n\in A).$ No generality is lost by assuming $\Pr(X_n\in A)>\Pr(Y_n\in A).$ Now suppose you've observed $Z_n$ for $100$ values of $n,$ and in every case you saw $Z_n\in A.$ What is the conditional probability that $Z_m\in A,$ where $m$ is some index not among those $100$ for which $Z_n$ was observed? The answer is that it is greater than the unconditional, or "marginal" probability that $Z_m\in A,$ because the observation all of those outcomes were in $A$ makes it more probable that $\Theta=1$ than it would otherwise have been. The terms in the sequence $\{Z_n\}$ are _conditionally independent given that $\Theta=0,$ or that $\Theta=1,$ but unconditionally they are not independent.

On the other hand the sequence $W$ is i.i.d. because for every value of $n$ you choose $\Theta_n,$ $X_n,$ and $Y_n,$ and consequently $W_n,$ independently of the results for all other values of $n,$ with the same probability distributions. Specifically $$ \Pr(W_n\in A) = \Pr(W_n\in A\mid\Theta=1)\Pr(\Theta=1) + \Pr(W_n\in A\mid\Theta=0)\Pr(\Theta=0) $$ and the event $W_n\in A$ has that same probability for each value of $n$ separately.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.