$3\sigma$ rule for multivariate normal distribution I was wondering if the $3\sigma$ rule that holds for 1D normal distribution also holds for multivariate normal distribution?
 A: It is often said that in high dimensions the probability distribution is concentrated away from the center.  So although in 1 D a 3 sigma interval will contain more than 99% of the distribution a three sigma circle for a 2D gaussian with iid components will contain less mass than the 1 D counterpart and the same for 3D compared to 2D etc.
A: Short answer: no, the rule does not hold in more dimensions.
In the general case (multivariate with arbitrary covariance matrix), the natural generalization of the "normalized distance from the mean", $d = |x -u|/\sigma$, is given by the Mahalanobis distance
$$d = \sqrt{ ({\bf x} - {\bf \mu})^t {\bf \Sigma}^{-1} ({\bf x} - {\bf \mu})}$$
Points of constant Mahalanobis distance lie on an ellipsoid.
If (and only if) the components are independent and with same variance, then $d=\frac{\|{\bf x} - {\bf \mu}\|}{\sigma}$.    
The threshold value that contains most (say, 99%) of the distribution varies with the dimension. Or, put in other way, the probability that $x$ takes a (Mahalanobis) distance less than (say) $d=3.0$ decreases with the dimension. 
This figure, taken from here ("Statistics for Imaging, Optics and Phtotononics", Peter Bajorski, fig. 5.21) (which explains all this in more detail), displays that probability as a function of the dimension, for distances $d=2.0$ and $d=3.0$ ("$2-$sigma" and "$3-$sigma").  

For example, we see that in 5 dimensions the probability that $x$ lies 'under 3 sigmas' is about $0.9$ (instead of $0.97$), and for '2 sigmas' is around $0.4$ (instead of $0.95$)
A: This is more a caveat than an answer (@leonbloy's is more then good enough), but I fell for this trap and would like others to avoid it.
In $D$ dimensions, the fraction of samples within Mahalanobis distance $3$ is NOT $0.997^D$.
This is wrong even if the covariance matrix is diagonal (even, the identity). Yes, a multivariate Gaussian is separable; and yes, along each principal axis the probability of being within three standard deviations is $0.997$.
However, the region within Mahalobonis distance $3$ is a hyperellipsoid (hypersphere), while the the intersection of the $\pm3\sigma$ 1D regions is a hyperrectangle (hypercube). The latter has an integral value of $0.997^D$, while the former is considerably smaller (and more difficult to compute).
A: As a small helper for all that want to find the generic $m$-sigma rules of $n$-dimensional hyper-ellipsoids:
Simply fire up wolframalpha and type in: 
$Q(n/2, 0, m^2/2)$. 
$Q$ stands for the generalized regularized incomplete gamma function.
For $n=1$ and $m=3$ you will obtain $0.997300...$,
for $n=2$ and $m=3$ the result is:
$0.988891...$ 
and for $n=3$ and $m=3$:
$0.970709...$. 
This answer is was inspired by the post of leonbloy.
