# Gaussian random vector covariance matrix calculation in two different ways

I'm trying to wrap my head around Gaussian random vectors, in particular their covariance matrices. I've seen two different definitions of a covariance matrix depending on whether you are calculating the matrix from observed results or from a mathematical model. If I let a Gaussian random vector be defined as such. $$X = \mu + AZ$$ Where $$\mu \in \mathbb{R}^n$$, $$A \in M_{n,k}$$ and $$Z = \{Z_0, ..., Z_k\}^T$$ where $$Z_i$$ are independent and identically distributed standard normal random variables.

I've seen that one definition of the covariance matrix $$\Sigma$$ is $$AA^T$$. And another is $$\Sigma_{i,j} = Cov[X_i, X_j] = E[(X_i-\mu_i)(X_j-\mu_j)]$$. I attempted to check that these are equivalent with an example. X = \begin{align}\begin{bmatrix}1 \\ 2\end{bmatrix} + \begin{bmatrix}1 & 2 \\ 3 & 4\end{bmatrix}\begin{bmatrix}Z_0 \\ Z_1\end{bmatrix}\end{align} I found, \Sigma = AA^T = \begin{align}\begin{bmatrix}5 & 11 \\ 11 & 25\end{bmatrix}\end{align} And, $$\Sigma_{0,1} = E[(X_0- \mu_0)(X_1 - \mu_1)] = E[(Z_0 + 2Z_1)(3Z_0+4Z_1)] = E[3Z_0^2 + 10Z_0Z_1 + 8Z_1^2]$$ Now I have no inclination as to if this makes any sense or why it might be true but I saw that if I let $$E[Z_iZ_j] = \delta_{i,j}$$ (Kronecker delta). Then we successfully find $$\Sigma_{0,1} = 11$$.

Could someone please provide more clarity on what is actually happening here?

• $E[Z_iZ_j] = \delta_{ij}$ is because $Z_0, Z_1$ are i.i.d. $N(0, 1)$. Commented Feb 16, 2023 at 21:51
• @Zhanxiong Ok, I think I understand why when $i \neq j$ the expectation is zero (both have mean value 0). And I understand that when $i = j$ the expected value would be positive. But it's exactly 1? Why? Commented Feb 16, 2023 at 22:11
• @repanda2236: Do you know if $Z \sim N(0, 1)$, then $E[Z^2] = 1$? Commented Feb 16, 2023 at 22:17
• @Zhanxiong No that's new to me, I'm trying to look up a proof but I can't seem to get the right search terms to find anything related. Commented Feb 16, 2023 at 22:21
• @repanda2236: By definition, if $Z \sim N(0, 1)$, then $E[Z] = 0, \operatorname{Var}(Z) = 1$, it then follows that $E[Z^2] = \operatorname{Var}(Z) + (E[Z])^2 = 1$. I am pretty sure if you checked the Wikipedia page of "normal distribution", you will find all these facts. Commented Feb 16, 2023 at 22:47