Let $Y_i$, $i = 1, \dotsc, n$ be random variables with binary values. How many parameters is needed parameterize the joint distribution $\Pr(Y_1 = y_1, \ldots, Y_n = y_n)$? What if $Y_i$ are all independent?

I am thinking that since each one has 2 possibilities and we have n of them, it should be $2^n$ different pairs. Am I on the right track?


Yes, you are quite close.

In general, you need $2^n-1$ parameters. We subtract $1$ as the probability sum to $1$.

If they are independent, then you have $n$ bernoulli distribution and only $n$ parameters is needed.

  • $\begingroup$ Can you elaborate on why we subtract 1? $\endgroup$ – user463768 Jul 16 '17 at 21:36
  • $\begingroup$ Let $n=2$. If we know $P(0,0), P(0,1), P(1,0)$, we are able to compute $P(1,1)$ isn't it? $\endgroup$ – Siong Thye Goh Jul 16 '17 at 21:37
  • $\begingroup$ I think I"m not understanding the idea of joint distribution correctly so I am still unsure why we subtract 1. Are you saying that we only need $2^n-1$ parameters because we can always calculate the last one from the rest we have? And for the second part, why do we have a bernoulli distribution just because $Y_i$ are all independent? $\endgroup$ – user463768 Jul 16 '17 at 22:03
  • $\begingroup$ Yes, you understand it correctly. Because we can compute the last entry due to the sum to $1$ constraint. For the case where things are independent, $Y_i$ are bernoulli right? Each Bernoulli distribution has $1$ parameter. $\endgroup$ – Siong Thye Goh Jul 16 '17 at 22:08
  • $\begingroup$ I don't think I learned it yet. But does independent means it would be bernoulli? $\endgroup$ – user463768 Jul 16 '17 at 22:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy