My understanding is that given matrix X, I can find its corresponding covariance matrix by:
- finding the means of each column
- subtracting each mean from each value in its respective column and
- multiplying the resulting matrix by its own transpose. Let's call this matrix C.
Here is what it would look like in Python:
Y = X - numpy.mean(X, axis = 0)
C = numpy.dot(Y, Y.T)
If I do this, I can prove mathematically (and experimentally using some simple Python code) that det(C) = 0 always.
However, a colleague tells me that using the inverse of a covariance matrix is common in his field and he showed me some R code to demonstrate.
I notice that R has several ways of calculating the covariance matrix that leads to different results. I also notice from Googling that some people say the covariance matrix is always singular (eg here) whereas others say it is not.
So, my question is: why the differences of opinion and what's the true answer?
EDIT: I discovered that the determinant is only zero if the matrix is square. If anybody knows the proof for this or can throw some further light on the matter, I'd be grateful.