I'm currently following this guide from Stanford's CS229 machine learning class for Factor Analysis. I followed through every point except for the following:

On bottom of page number 5, following proof is given: enter image description here

Here, what I don't understand is that how expectation of $zz^{T}$ is identity matrix $I$? In the notes it's written that since $z$ ~ $\mathcal{N}(0, I)$, $E[zz^{T}] = Cov(z) =I$. But I can't figure out the relation between Co-variance of $z$ and it's expectation.

It might be very trivial question but I can't find any explanation on this. Any help would be appreciated.

EDIT Also in next proof, there is same case for $\epsilon$ as mentioned above (here covariance matrix of $\epsilon$ is $\psi$ and in above case it was identity matrix $I$). (on starting of page number 6). So is there any thumb rule for such cases?


1 Answer 1


The formula for covariance can be given by:

$cov(X,Y) = E\left(X - E(X)(Y - E(Y)\right)$

which is the same as:

$cov(X,Y) = E(XY) - E(X)E(Y)$

(the derivation can be found here: https://en.wikipedia.org/wiki/Covariance)

In your case, $X=Y=Z$ and $E(Z) = 0$

Then $cov(Z,Z) = E(ZZ)$

With random vectors it is pretty much the same thing, you just need to transpose before multiplication (also in the wikipedia page).

Also, $cov(Z) = I$ because $z \sim N(0,I)$

  • $\begingroup$ so you mean $cov(z, z^{T}) = cov(z) = cov(z^{T}) = cov(z, z) = I$? $\endgroup$
    – Kaushal28
    Jun 29, 2018 at 12:51
  • $\begingroup$ Yes, remember that $cov(z,z) = var(z)$. cov(z) is just a lazy way of writing cov(z,z), usually. $\endgroup$ Jun 29, 2018 at 12:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .