I actually was perusing here right now to see if anything could explain a result I have been getting - of a covariance matrix which has a negative eigenvalue, yet the correlation matrix does not - but got a big surprise when I see all these statements that covariance matrices are always positive semidefinite.
I have certainly not been getting this result. I get LOTS of negative eigenvalues when using the covariance matrix of things that are pairwise at least rather correlated. For instance, the correlation between various exponentially weighted averages of random walks with different time constants. I only need to get into the 30s of them with time constants increasing in a geometric sequence with a factor of 1.04 (in other words, 1, 1.04, 1.04^2) and BAM. Negative eigenvalues. Very SMALL negative eigenvalues, like 10^-6, but negative. And it's not matlab calculating eigenvalues and eigenvectors wrong either, I try premultiplying and postmultiplying the matrix with an eigenvector for a negative eigenvalue and the result is a negative number. And don't even get me started on covariance matrices derived from actual observations rather than abstract functions, so that the "observed" covariance and correlation, which is the covariance divided by the square root of the corresponding variances of the 2 variables it's a covariance between, is not the underlying covariance or correlation of the actual random variable. In other words, say you have a particle moving around, and it has some mean square change in its position after a given amount of time. When the value which is observed over a period of time is used as the actual value - for instance, the mean square amount it moved in a millisecond, averaged over a period of a second - then just by luck it may not be the real mean square change in position after a millisecond, maybe it moved a little more or less than usual over that second. Negative eigenvalues are just frigging everywhere. So what is the deal then? Is it all numerical subtraction error? If I did it all with enough precision, would matlab find it to be positive definite always, when there is nothing linearly dependent (eigenvalues can be 0, but not negative)?