Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Consider a random column vector $\mathbf{x}$, of dimension $m$. That is, it is a random vector, composed of $m$ random variables. The PDF of the random vector $\mathbf{x}$ is thus the joint-PDF of its $m$ random variable components.

Let us assume that one wishes to whiten the random variables of this vector. That is, you want to de-correlate them, and then scale their variances to be unity. One way to do this is to compute a whitening matrix $\mathbf{V_{\mathrm{mxm}}}$, and one way to compute the whitening matrix is by:

$$ \mathbf{V_{\mathrm{mxm}}} = \mathbf{E} \mathbf{D^{\mathrm{-\frac{1}{2}}}}\mathbf{E^{T}} $$

where $\mathbf{E}$ is the column wise eigen-vector matrix, and $\mathbf{D}$ its corresponding eigen-value matirx, coming from the eigenvector decomposition of x's covariance matrix, $\mathbf{R_{\mathrm{xx}}} = \mathbf{E}\mathbf{D}\mathbf{E^T}$. The $\mathbf{D^{-\frac{1}{2}}}$ implies the " inverse matrix square root". In fact there are many other types of whitening matricies that can be constructed.

My question is, to my knowledge, any construction of a whitening matrix needs to have an inverse operation within it. For me, this is fine if my dimensionality is 'small', but I hesitate to use this method for larger dimensions of $m$.

What other methods of computing a whitening matrix might exist that do not involve the computation of inverses?

Much obliged.

share|improve this question

1 Answer 1

up vote 2 down vote accepted

Let $X$ be the original random with covariance matrix $\Sigma$. Since $\Sigma$ is Hermitian symmetric, we can do an eigen-value decomposition with an orthonormal basis:

$\Sigma = E \Lambda E^T$

Now, the whitening matrix is $W = (E \Lambda^{1/2})^{-1}$ since covariance matrix of $WX$ is $I$. The good thing about having an orthonormal basis is that $E^{-1} = E^T$. Hence, $W = \Lambda^{-1/2} E^{T}$. Hence, you only need to take inverse of the diagonal matrix, which is a matrix with inverse of the original diagonal entries. This is not computationally expensive at all.

share|improve this answer
    
Ah yes, you are correct. Would this then be considered the most efficient methods VS something like taking the inverse square root of the correlation matrix itself, or the inverse of the cholesky decomposition? –  Mohammad Aug 21 '12 at 18:59
    
Yes, I think so. –  Kartik Audhkhasi Aug 21 '12 at 19:37
    
Isn't computing the entire eigenvector matrix also expensive, even for a symmetric matrix? I wonder if Cholesky itself isn't faster. –  Gabriel Landi Aug 21 '12 at 21:41
    
@GabrielLandi Thats a good/concerning point actually. Its clear that the computation of the inverse square root of a diagonal matrix is O(N), (and so, I dont worry about it), but I, like you, am concerned about having to compute the eigen-vector decomposition on a large matrix, in order to compute D to begin with. Thoughts? –  Mohammad Aug 21 '12 at 22:07
    
If your random vector is wide-sense stationary (WSS), its covariance matrix will be a circulant. The eigenvectors of a circulant matrix are the discrete Fourier transform (DFT) basis vectors. So you don't need to compute them via an eigenvector decomposition, saving you all computation. Will update if I can think of any other simplifying cases. –  Kartik Audhkhasi Aug 21 '12 at 23:52

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.