Covariance of a random vector

Question

If $Y_1,..Y_n$ are independent random variables, how do I work out

cov$\begin{pmatrix}Y_1\\.\\.\\.\\Y_n \end{pmatrix}$?

The covariance of the vector

Answer 1

If the variables are independent, then $Cov(Y_i,Y_j)=0$ for $i\neq j$.

So the covariance matrix is diagonal and the $i$-th diagonal term is $Var(Y_i)$.

---

If $X$ and $Y$ are random variables, you can calculate its covariance defined by $Cov(X,Y)=E[(X-E(X))(Y-E(Y))]$. Note that $Cov(X,X)=Var(X)$.

The generalization for a random vector of the variance of a random variable is a matrix called the covariance matrix of the vector, or variance-covariance matrix. It's the matrix $(\Sigma_{ij})$ with $\Sigma_{ij}=Cov(X_i,X_j)$. So its diagonal entries are the variances. If $X=(X_1,\dots,X_n)^T$ is a random vector, its variance-covariance matrix is denoted by $Var(X)$ or sometimes $Cov(X)$. You also have $V(X)=E[(X-E(X))(X-E(X))^T]$. Here $E(X)$ is the vector $(E(X_1),\dots,E(X_n))^T$.

So if you write $Cov(X,Y)$ I understand it as $E((X-E(X))(Y-E(Y))]$ : it's a number, not a matrix.

If you write $Cov\begin{pmatrix}X\\Y\end{pmatrix}$, I understand this as the covariance matrix of the random vector $(X,Y)^T$, which is $$\begin{pmatrix}Var(X) & Cov(X,Y)\\ Cov(X,Y) & Var(Y)\end{pmatrix}$$

Last thing, we also have a notion of covariance of two random vectors $X$ and $Y$, which is defined as $Cov(X,Y)=E[(X-E(X))(Y-E(Y))^T]$. Note that if $Y=X$, we get the variance-covariance matrix of $X$. So we fully generalized the 1-dimensional case.

I hope I didn't make you more confused about that.

Answer 2

$\text{cov}(Y) = E\left((Y-EY)(Y-EY)^T \right) = E \begin{bmatrix} (Y_1 - EY_1)^2& ... & (Y_1-EY_1)(Y_n-EY_n) \\ ... & ...\\ (Y_n - EY_n)(Y_1 - EY_1)&... & (Y_n-EY_n)^2 \end{bmatrix} = \begin{bmatrix} \text{var}(Y_1)&... & 0 \\ 0 & ...& 0\\ 0&... & \text{var}(Y_n) \end{bmatrix} =\text{diag}(\sigma^2_{Y_1},..., \sigma^2_{Y_n}) $