# Stuck while understanding derivation of Factor Analysis

I'm currently following this guide from Stanford's CS229 machine learning class for Factor Analysis. I followed through every point except for the following:

On bottom of page number 5, following proof is given:

Here, what I don't understand is that how expectation of $zz^{T}$ is identity matrix $I$? In the notes it's written that since $z$ ~ $\mathcal{N}(0, I)$, $E[zz^{T}] = Cov(z) =I$. But I can't figure out the relation between Co-variance of $z$ and it's expectation.

It might be very trivial question but I can't find any explanation on this. Any help would be appreciated.

EDIT Also in next proof, there is same case for $\epsilon$ as mentioned above (here covariance matrix of $\epsilon$ is $\psi$ and in above case it was identity matrix $I$). (on starting of page number 6). So is there any thumb rule for such cases?

The formula for covariance can be given by:

$cov(X,Y) = E\left(X - E(X)(Y - E(Y)\right)$

which is the same as:

$cov(X,Y) = E(XY) - E(X)E(Y)$

(the derivation can be found here: https://en.wikipedia.org/wiki/Covariance)

In your case, $X=Y=Z$ and $E(Z) = 0$

Then $cov(Z,Z) = E(ZZ)$

With random vectors it is pretty much the same thing, you just need to transpose before multiplication (also in the wikipedia page).

Also, $cov(Z) = I$ because $z \sim N(0,I)$

• so you mean $cov(z, z^{T}) = cov(z) = cov(z^{T}) = cov(z, z) = I$? – Kaushal28 Jun 29 '18 at 12:51
• Yes, remember that $cov(z,z) = var(z)$. cov(z) is just a lazy way of writing cov(z,z), usually. – mechanical_fan Jun 29 '18 at 12:59