After getting my answer in this:
I am running into a problem which I do not understand. I have n dependent gaussian random variables that are related by a known n x n covariance matrix C. The solution I was given was to take the positive definite square root of the inverse of C - which means I put the n independent random variables (call them the vector x) through a linear transformation (D^-.5*Q)*x, where D is a square diagonal matrix of the eigenvalues (of C) and Q's rows are the corresponding eigenvectors.
The trouble is, the covariance matrix isn't necessarily positive definite under certain values of the parameters of the problem. I don't want a complex matrix, taking the negative eigenvalues to the -.5 power, so my idea was, I keep just the positive eigenvalues. So I look at D, and if it's a 100 x 100 matrix but only 67 of the eigenvalues are positive, I throw out the negative ones, and use only a 67 x 67 matrix for D, which I can take to the -.5 power, and then Q is no longer a square matrix but a 67 x 100 matrix, and the 100-length vectors are the normalized eigenvectors of C corresponding to the eigenvalues that I kept. And lo and behold, when I find this matrix A=(D^-.5*Q), and I put x under the linear transformation Ax, so that the covariance matrix of the result SHOULD be a 67 x 67 matrix, for 67 random variables that are actually variance-1 and independent and constructed out of the original 100 random variables that were too highly correlated for their own good, and I calculate what SHOULD be the new covariance matrix ACA', I get that it is very nearly the 67 x 67 entry identity matrix. GREAT.
it does not hold up to actual observation. When I actually try this transformed set of random variables, they are not even CLOSE to being independent. But apparently, only when I throw out some negative eigenvalues, or when it's really close to not being positive definite. In other words, if the parameters of the problem are such that all 100 eigenvalues are quite positive, it works - Ax produces 100 random variables that appear, by monte carlo simulations, to truly be independent. But as soon as I throw away any eigenvalues - and I don't even have to do that, as soon as the parameters of the problem put it CLOSE to being at the point where I would be getting negative eigenvalues, suddenly different entries in Ax start exhibiting absurdly high correlations. 0.6, 0.7, as high as 0.9. Even though ACA' is very nearly an identity matrix, albeit of fewer dimensions than the original C. I thought at first that maybe I wasn't doing enough tests, but that thought is gone - I try it with tens of thousands of trials and the correlation of something that was 0.709 before maybe changes to 0.706 with the completely new data. They ARE highly correlated. I thought for a while that it was a result of small numerical errors in dealing with large matrices. But that doesn't explain why it suddenly happens when the parameters of the problems contrive to make the lowest eigenvalue close to crossing below 0.
Am I missing something? Isn't ACA' ALWAYS supposed to be the covariance matrix of Ax if C is the covariance of x? Is it ever not? Because ACA' here is very nearly an identity matrix and Ax's elements are flagrantly dependent. And if not, is there another method of producing as many independent equal variance random variables as possible besides throwing out the negative eigenvalues as I was doing?