mean and variance normalization of vectors I have vectors $x \in \mathbb{R}^n$ and I expect some multivariate normal distribution.
I want to normalize the vectors in such a way that $y = M (x - b)$ has mean zero ($\operatorname{E}[Y] = 0$) and variance 1 ($\operatorname{var}(Y) = 1_n$).
For $n=1$, that is basically the Standard score.


*

*With $M = 1_n$ and $b = \operatorname{E}[X]$, I get it mean-normalized but not variance-normalized.

*With $M$ being a non-negative symmetric square root of $\operatorname{var}(X)^{-1}$, we get $\operatorname{var}(Y) = 1_n$. We can choose $b$ as in case 1.
I want to estimate $M$ and $b$ based on some running/moving algorithm. For the first case, this is easy:
For every new $x$, I can use the update rule
$$b \leftarrow x \alpha + b (1 - \alpha)$$
for some $\alpha$. I could set $\alpha = 1/N$ and set $N \leftarrow N + 1$ in every iteration, starting with $N = 1$.
How would I do something similar for the second case, esp. for $M$?
I could use a similar update rule to calculate the covariance matrix, e.g.
$$\Sigma \leftarrow (x - b) (x - b)^T \alpha + \Sigma (1- \alpha) $$
but I'm not sure how well that works in practice (also, I haven't really seen that formula in such form) or if that even converges (given that it uses also the moving $b$).
Also, if I use this, I only have an estimation of $\operatorname{var}(X)$ but I still need the $M$. I probably could use another estimation to get the $M$ based on the $\Sigma$ (which one?), but I wonder if I could just get an estimation of $M$ more directly.

(I asked the same question also on MetaOptimize.)
 A: I assume you want to avoid matrix inversion at each step? There are two possibilities:
I. Estimate inverse covariance directly
If you average the inverse of X*X^T instead of taking the inverse of the average of X*XT, then y will not be a normal random variable. Still, it can asymptotically look like one. See e.g.:
P. Shaman, "An approximate inverse for the covariance matrix of moving average and autoregressive processes"
http://projecteuclid.org/download/pdf_1/euclid.aos/1176343085
II. Estimate covariance and invert with iterative method
Estimate covariance in the usual way:
Using homogeneous coordinates Z = [x0, x1, .., xn, 1]
Keep a running weighted sum of S = Z * Z^T
The mean        Bj = Snj / Snn for    j < n
The covariance Cij = Sij / Snn for i, j < n
Then since Cij changes slowly over time, compute at each step the inverse covariance M = Inv(C) given a fast iterative method by remembering the prior Mprev, e.g.
F. Soleymani, "On a fast iterative method for approximate inverse of matrices"
http://www.mathnet.or.kr/mathnet/thesis_file/CKMS-28-2-407-418.pdf
