I have vectors $x \in \mathbb{R}^n$ and I expect some multivariate normal distribution.
I want to normalize the vectors in such a way that $y = M (x - b)$ has mean zero ($\operatorname{E}[Y] = 0$) and variance 1 ($\operatorname{var}(Y) = 1_n$).
For $n=1$, that is basically the Standard score.
With $M = 1_n$ and $b = \operatorname{E}[X]$, I get it mean-normalized but not variance-normalized.
With $M$ being a non-negative symmetric square root of $\operatorname{var}(X)^{-1}$, we get $\operatorname{var}(Y) = 1_n$. We can choose $b$ as in case 1.
I want to estimate $M$ and $b$ based on some running/moving algorithm. For the first case, this is easy:
For every new $x$, I can use the update rule
$$b \leftarrow x \alpha + b (1 - \alpha)$$
for some $\alpha$. I could set $\alpha = 1/N$ and set $N \leftarrow N + 1$ in every iteration, starting with $N = 1$.
How would I do something similar for the second case, esp. for $M$?
I could use a similar update rule to calculate the covariance matrix, e.g.
$$\Sigma \leftarrow (x - b) (x - b)^T \alpha + \Sigma (1- \alpha) $$
but I'm not sure how well that works in practice (also, I haven't really seen that formula in such form) or if that even converges (given that it uses also the moving $b$).
Also, if I use this, I only have an estimation of $\operatorname{var}(X)$ but I still need the $M$. I probably could use another estimation to get the $M$ based on the $\Sigma$ (which one?), but I wonder if I could just get an estimation of $M$ more directly.
(I asked the same question also on MetaOptimize.)