2
$\begingroup$

I'm doing some basic error, but I just can't see where...

Let $X_i\sim \mathrm{Bin}(\theta_i,n)$, and $X_j\sim \mathrm{Bin}(\theta_j,n)$

I want to find $\mathrm{Cov}(X_i,X_j)$.

So,

$E(X_iX_j)=E(E(X_iX_j|X_i=x_i))=E(X_i n \theta_j)=n\theta_i n\theta_j$. But this isn't correct is it? Why?

$\endgroup$
  • 1
    $\begingroup$ We need information about any possible relationship between $X_i$ and $X_j$. If they are independent, the covariance is $0$. $\endgroup$ – André Nicolas Jul 9 '14 at 21:53
  • $\begingroup$ @AndréNicolas, thanks for your interest in this question. The only thing I know is that $\mathbf{X}=(X_1,...,X_k)$, where $\mathbf{X} \sim Multi (n,\mathbf{\Theta})$ $\endgroup$ – An old man in the sea. Jul 9 '14 at 21:54
  • $\begingroup$ So, the marginal distribution of a multinomial is a binomial. $\endgroup$ – An old man in the sea. Jul 9 '14 at 22:07
  • $\begingroup$ If you are interested in the distribution of the number of Type $j$, then yes. $\endgroup$ – André Nicolas Jul 9 '14 at 22:10
  • 1
    $\begingroup$ The binomials are in this case not independent. $\endgroup$ – André Nicolas Jul 9 '14 at 22:20
1
$\begingroup$

Recall that $X=(X_i)_i$ multinomial $(n,(\theta_i)_i)$ with $\sum\limits_i\theta_i=1$ can be realized as $$X_i=\sum\limits_{k=1}^n\mathbf 1_{U_k=i},$$ for every $i$, where $(U_k)$ is i.i.d. with $P(U_k=i)=\theta_i$ for every $k$ and every $i$. Thus, for every $i\ne j$, $$ X_iX_j=\sum_k\mathbf 1_{U_k=i}\mathbf 1_{U_k=j}+\sum_{k\ne\ell}\mathbf 1_{U_k=i}\mathbf 1_{U_\ell=j}.$$ Since $[U_k=i]\cap[U_k=j]=\varnothing$ for every $k$ and the events $[U_k=i]$ and $[U_\ell=j]$ are independent for every $k\ne\ell$, this yields $$ E(X_iX_j)=\sum_{k\ne\ell}P(U_k=i)P(U_\ell=j)=n(n-1)\theta_i\theta_j. $$ Since $E(X_i)=n\theta_i$ and $E(X_j)=n\theta_j$, one gets $$ \mathrm{Cov}(X_i,X_j)=n(n-1)\theta_i\theta_j-n\theta_in\theta_j=-n\theta_i\theta_j.$$

$\endgroup$
1
$\begingroup$

The covariance should be $E(X_iX_j - E(X_i)E(X_j))$, which in your calculations could be $0$ when $i \not = j$.

But you seem to have said $E(X_j|X_i=x_i)= n\theta_j$. Are you sure? What is $E(X_j|X_i=n)$?

$\endgroup$
  • $\begingroup$ many thanks for your pedagogical perspective. Afterwards I decided to go with the moment generating function... Just to be sure, $E(X_j|X_i=n)=0$, right? $\endgroup$ – An old man in the sea. Jul 10 '14 at 10:17
  • $\begingroup$ @An old man in the sea: Yes - if everything is type $j$ then nothing can be of type $i$. $\endgroup$ – Henry Jul 10 '14 at 10:21
1
$\begingroup$

From a comment by the OP, the real question is as follows. We repeat an experiment independently $n$ times. The possible outcomes are $A_1, A_2, \dots, A_q$ with probabilities $\theta_1,\theta_2,\dots,\theta_q$. Let $X_i$ be the number of Type $A_i$ outcomes in the $n$ trials. We want $\text{Cov}(X_i,X_j)$. Informally, the covariance is negative: if $A_i$ occurs lotsa times, then probably $X_j$ will not occur many times.

So we want $E(X_iX_j)-E(X_i)E(X_j)$. Te expectations $E(X_i)$ and $E(X_j)$ are easy. We need to find $E(X_iX_j)$.

Let $U_k=1$ if on the $k$-th trial we get $A_i$. Let $V_k$ be defined similarly for $A_j$. We want $E((U_1+\cdots+U_n)(V_1+\cdots +V_n))$. By the linearity of expectation, and the fact that $U_sV_s=0$ and $\Pr(U_sV_t=1)=\theta_i\theta_j$ if $s\ne t$, we get that $E((U_1+\cdots+U_n)(V_1+\cdots +V_n))=n(n-1) \theta_i\theta_j$.

But $E(X_i)=n\theta_i$, with a similar expression for $E(X_j)$. Subtract. We get the desired result.

$\endgroup$
  • $\begingroup$ Many thanks for your help. $\endgroup$ – An old man in the sea. Jul 10 '14 at 10:19
  • $\begingroup$ You are welcome. $\endgroup$ – André Nicolas Jul 10 '14 at 12:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.