IF I have $X,Y_1,...,Y_n$ iid then how do I calculate:

cov $\left [\begin{pmatrix}X\\.\\.\\.\\X \end{pmatrix}, \begin{pmatrix}Y_1\\.\\.\\.\\Y_n \end{pmatrix}\right]$?


This is known as the cross-covariance between vectors, and is defined by $$ \text{cov}[\boldsymbol{X},\boldsymbol{Y}] = \text{E}[(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T}] $$

where $$ \boldsymbol{\mu_X} = \text{E}[\boldsymbol{X}]\\ \boldsymbol{\mu_Y} = \text{E}[\boldsymbol{Y}] $$

In your case, because all the components of $\boldsymbol{X}$ are the same, things simplify greatly.

$$ \boldsymbol{X} = X \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right], \;\; \boldsymbol{\mu_X} = \mu_X \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right] $$ Where $\mu_X=\text{E}[X]$. Then $$ \boldsymbol{X}-\boldsymbol{\mu_X} = (X-\mu_X) \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right] $$ Now $$ (\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T} = (X-\mu_X) \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right]\left[ \begin{array}{cccc}Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n\end{array} \right] $$ where $\mu_m=\text{E}[Y_m]$ for $m\in[1,2,\cdots,n]$. Expanding out that matrix product we have $$ (\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T} = (X-\mu_X)\left[ \begin{array}{cccc} Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n\\ Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n\\ \vdots&\vdots&\ddots&\vdots\\ Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n \end{array} \right] $$

Taking that scalar inside the matrix, we see it multiplies each entry in the matrix. Then taking the expectation of the result finally gives $$ \text{E}[(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T}] = \left[ \begin{array}{cccc} \text{E}[(X-\mu_X)(Y_1-\mu_1)]&\text{E}[(X-\mu_X)(Y_2-\mu_2)]&\cdots&\text{E}[(X-\mu_X)(Y_n-\mu_n)]\\ \text{E}[(X-\mu_X)(Y_1-\mu_1)]&\text{E}[(X-\mu_X)(Y_2-\mu_2)]&\cdots&\text{E}[(X-\mu_X)(Y_n-\mu_n)]\\ \vdots&\vdots&\ddots&\vdots\\ \text{E}[(X-\mu_X)(Y_1-\mu_1)]&\text{E}[(X-\mu_X)(Y_2-\mu_2)]&\cdots&\text{E}[(X-\mu_X)(Y_n-\mu_n)] \end{array} \right] $$ $$ = \left[ \begin{array}{cccc} \text{cov}(X,Y_1)&\text{cov}(X,Y_2)&\cdots&\text{cov}(X,Y_n)\\ \text{cov}(X,Y_1)&\text{cov}(X,Y_2)&\cdots&\text{cov}(X,Y_n)\\ \vdots&\vdots&\ddots&\vdots\\ \text{cov}(X,Y_1)&\text{cov}(X,Y_2)&\cdots&\text{cov}(X,Y_n) \end{array} \right] $$

Now we are at the answer: you specified all the variables to be identically distributed and independent. Independent variables have covariance $0$. SO, you get the all zeros matrix for your answer $$ \text{cov}(\boldsymbol{X},\boldsymbol{Y})=\text{E}[(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T}] = \left[ \begin{array}{cccc} 0&0&\cdots&0\\ 0&0&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&0\\ \end{array} \right] $$

  • 3
    $\begingroup$ "The very definition of independent variables is that their covariance is $0$" is completely wrong. Independence implies zero covariance, but variables with zero covariance are not necessarily independent. You have confused "independent" with "uncorrelated". $\endgroup$ – Anon Apr 24 '16 at 10:53
  • $\begingroup$ Thanks for the correction. $\endgroup$ – rajb245 Apr 26 '16 at 3:21

Covariance of $2$ vectors is basically what is called a variance-covariance matrix $(\Sigma)$ defined as $$((\Sigma_{ij}))=Cov(X_i,Y_j)$$ where $Cov(A,B)=E(AB)-E(A)E(B)$

For more details, just Google Variance Covariance matrix.

Specifically, because of the iid character of your variables, $Cov$ will be $0$ for all.

  • 1
    $\begingroup$ The variance covariance matrix is for one vector containing multiple random variables. Thia is for two vectors. $\endgroup$ – usainlightning Apr 18 '16 at 17:25
  • 1
    $\begingroup$ @usainlightning : Please read the 2nd paragraph of the Wikipedia article on Covariance matrix before proceeding. $\endgroup$ – Qwerty Apr 18 '16 at 20:26
  • 1
    $\begingroup$ actually I agree with Qwerty, with the precision that we don't really care about the two random vectors... One could concatenate the two random vectors together and compute the classical covariance matrix of a random vector. $\endgroup$ – Marine Galantin Nov 12 '19 at 1:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.