31
$\begingroup$

I was in a seminar today and the lecturer said that the gaussian distribution is isotropic. What does it mean for a distribution to be isotropic? It seems like he is using this property for the pseudo-independence of vectors where each entry is sampled from the normal distribution.

$\endgroup$
  • 2
    $\begingroup$ In general, a multivariate normal distribution can be anisotropic depending on the covariance matrix. There has. clearly been some miscommunication somewhere along the way. $\endgroup$ – Brian Borchers Oct 30 '16 at 18:52
51
$\begingroup$

TLDR: An isotropic gaussian is one where the covariance matrix is represented by the simplified matrix $\Sigma = \sigma^{2}I$.

Some motivations:

Consider the traditional gaussian distribution:

$$ \mathcal{N}(\mu,\,\Sigma) $$

where $\mu$ is the mean and $\Sigma$ is the covariance matrix.

Consider how the number of free parameters in this Gaussian grows as the number of dimensions grows.

$\mu$ will have a linear growth. $\Sigma$ will have a quadratic growth!

This quadratic growth can be very computationally expensive, so $\Sigma$ is often restricted as $\Sigma = \sigma^{2}I$ where $\sigma^{2}I$ is a scalar variance multiplied by an identity matrix.

Note that this results in $\Sigma$ where all dimensions are independent and where the variance of each dimension is the same. So the gaussian will be circular/spherical.

Disclaimer: Not a mathematician, and I only just learned about this so may be missing some things :)

Hope that helps!

$\endgroup$
  • 1
    $\begingroup$ The variables of the multivariate Gaussian may not be independent, even if they have zero covariance. Covariance is a measure of only linear association. Independence implies zero covariance but not vice versa. See this example. $\endgroup$ – mloning Nov 29 '19 at 9:10
7
$\begingroup$

I'd just like to add a bit of visuals to the other answers.

When the variables are independent, i.e. the distrubtion is isotropic, it means that the distribution is aligned with the axis.

For example, for $\Sigma = \begin{pmatrix}1 & 0 \\ 0 & 30\end{pmatrix}$, you'd get something like this:

image of 2D gaussian with higher Y variance

So, what happens when it is not isotropic? For example, when $\Sigma = \begin{pmatrix}1 & 15 \\ 15 & 30\end{pmatrix}$, the distribution appears "rotated", no longer aligned with the axes:

image of 2D gaussian with covariance between X and Y

Note that this is just an example, the $\Sigma$ above is invalid since it is not PSD.


Code:

import numpy as np
from matplotlib import pyplot as plt

pts = np.random.multivariate_normal([0, 0], [[1,15],[15,31]], size=10000, check_valid='warn')

plt.scatter(pts[:, 0], pts[:, 1], s=1)
plt.xlim((-30,30))
plt.ylim((-30,30))
$\endgroup$
5
$\begingroup$

I am not a math major student but I will give a try to describe my understanding: an isotropic gaussian distribution means a multidimensional gaussian distribution with its variance matrix as an identity matrix multiplied by the same number on its diagonal. Each dimension can be seen as an independent one-dimension gaussian distribution (no covariance exists).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.