# Covariance of two random vectors

IF I have $X,Y_1,...,Y_n$ iid then how do I calculate:

cov $\left [\begin{pmatrix}X\\.\\.\\.\\X \end{pmatrix}, \begin{pmatrix}Y_1\\.\\.\\.\\Y_n \end{pmatrix}\right]$?

This is known as the cross-covariance between vectors, and is defined by $$\text{cov}[\boldsymbol{X},\boldsymbol{Y}] = \text{E}[(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T}]$$

where $$\boldsymbol{\mu_X} = \text{E}[\boldsymbol{X}]\\ \boldsymbol{\mu_Y} = \text{E}[\boldsymbol{Y}]$$

In your case, because all the components of $\boldsymbol{X}$ are the same, things simplify greatly.

$$\boldsymbol{X} = X \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right], \;\; \boldsymbol{\mu_X} = \mu_X \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right]$$ Where $\mu_X=\text{E}[X]$. Then $$\boldsymbol{X}-\boldsymbol{\mu_X} = (X-\mu_X) \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right]$$ Now $$(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T} = (X-\mu_X) \left[ \begin{array}{c}1\\1\\\vdots\\1\end{array} \right]\left[ \begin{array}{cccc}Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n\end{array} \right]$$ where $\mu_m=\text{E}[Y_m]$ for $m\in[1,2,\cdots,n]$. Expanding out that matrix product we have $$(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T} = (X-\mu_X)\left[ \begin{array}{cccc} Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n\\ Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n\\ \vdots&\vdots&\ddots&\vdots\\ Y_1-\mu_1&Y_2-\mu_2&\cdots&Y_n-\mu_n \end{array} \right]$$

Taking that scalar inside the matrix, we see it multiplies each entry in the matrix. Then taking the expectation of the result finally gives $$\text{E}[(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T}] = \left[ \begin{array}{cccc} \text{E}[(X-\mu_X)(Y_1-\mu_1)]&\text{E}[(X-\mu_X)(Y_2-\mu_2)]&\cdots&\text{E}[(X-\mu_X)(Y_n-\mu_n)]\\ \text{E}[(X-\mu_X)(Y_1-\mu_1)]&\text{E}[(X-\mu_X)(Y_2-\mu_2)]&\cdots&\text{E}[(X-\mu_X)(Y_n-\mu_n)]\\ \vdots&\vdots&\ddots&\vdots\\ \text{E}[(X-\mu_X)(Y_1-\mu_1)]&\text{E}[(X-\mu_X)(Y_2-\mu_2)]&\cdots&\text{E}[(X-\mu_X)(Y_n-\mu_n)] \end{array} \right]$$ $$= \left[ \begin{array}{cccc} \text{cov}(X,Y_1)&\text{cov}(X,Y_2)&\cdots&\text{cov}(X,Y_n)\\ \text{cov}(X,Y_1)&\text{cov}(X,Y_2)&\cdots&\text{cov}(X,Y_n)\\ \vdots&\vdots&\ddots&\vdots\\ \text{cov}(X,Y_1)&\text{cov}(X,Y_2)&\cdots&\text{cov}(X,Y_n) \end{array} \right]$$

Now we are at the answer: you specified all the variables to be identically distributed and independent. Independent variables have covariance $0$. SO, you get the all zeros matrix for your answer $$\text{cov}(\boldsymbol{X},\boldsymbol{Y})=\text{E}[(\boldsymbol{X}-\boldsymbol{\mu_X})(\boldsymbol{Y}-\boldsymbol{\mu_Y})^\text{T}] = \left[ \begin{array}{cccc} 0&0&\cdots&0\\ 0&0&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&0\\ \end{array} \right]$$

• "The very definition of independent variables is that their covariance is $0$" is completely wrong. Independence implies zero covariance, but variables with zero covariance are not necessarily independent. You have confused "independent" with "uncorrelated".
– Anon
Apr 24, 2016 at 10:53
• Thanks for the correction. Apr 26, 2016 at 3:21

Covariance of $2$ vectors is basically what is called a variance-covariance matrix $(\Sigma)$ defined as $$((\Sigma_{ij}))=Cov(X_i,Y_j)$$ where $Cov(A,B)=E(AB)-E(A)E(B)$

For more details, just Google Variance Covariance matrix.

Specifically, because of the iid character of your variables, $Cov$ will be $0$ for all.

• The variance covariance matrix is for one vector containing multiple random variables. Thia is for two vectors. Apr 18, 2016 at 17:25
• @usainlightning : Please read the 2nd paragraph of the Wikipedia article on Covariance matrix before proceeding. Apr 18, 2016 at 20:26
• actually I agree with Qwerty, with the precision that we don't really care about the two random vectors... One could concatenate the two random vectors together and compute the classical covariance matrix of a random vector. Nov 12, 2019 at 1:05