88 votes
Accepted

Gaussian distribution is isotropic?

TLDR: An isotropic gaussian is one where the covariance matrix is represented by the simplified matrix $\Sigma = \sigma^{2}I$. Some motivations: Consider the traditional gaussian distribution: $$ \...
user avatar
  • 996
66 votes
Accepted

Why we consider log likelihood instead of Likelihood in Gaussian Distribution

It is extremely useful for example when you want to calculate the joint likelihood for a set of independent and identically distributed points. Assuming that you have your points: $$X=\{x_1,x_2,\ldots,...
user avatar
  • 1,192
44 votes
Accepted

A Mathematical Paradox About Probabilities

Something to think about: Since the coin flips are independent, and assuming the coin is fair, the probability that ten coin flips land heads is: $$P(H)\cdot P(H)\cdot P(H)\cdot P(H)\cdot P(H)\cdot ...
user avatar
  • 6,337
25 votes
Accepted

Basic Calc - why can't I solve this integral?

When you take the derivative of $$\frac{1}{2x}e^{x^{2}}$$ you need to use the product and chain rules. You get $$\frac{d}{dx}\frac{1}{2x}e^{x^{2}} = -\frac{1}{2x^{2}}e^{x^{2}}+\frac{1}{2x}2xe^{x^{2}} ...
user avatar
  • 9,159
23 votes

Product of Two Multivariate Gaussians Distributions

An alternative expression of the PDF proportional to the product is: $\Sigma_3 = \Sigma_1(\Sigma_1 + \Sigma_2)^{-1}\Sigma_2$ $\mu_3 = \Sigma_2(\Sigma_1 + \Sigma_2)^{-1}\mu_1 + \Sigma_1(\Sigma_1 + \...
user avatar
21 votes

Scaling the normal distribution?

On the first page of the cited document, $X_1$ and $X_2$ were previously defined to be two (distinct) independent, identically distributed random variables. For your purposes, the "identically ...
user avatar
  • 86.6k
19 votes

A Mathematical Paradox About Probabilities

All the answers provided explain why there is no "paradox." I would like to provide you an answer that is rather intuitive (hopefully) than formal. Your question is a good example of what is known as ...
user avatar
  • 1,157
18 votes

Is the product of two Gaussian random variables also a Gaussian?

As pointed out by Davide Giraudo, the characteristic function of Z is $\varphi_Z (t) = \frac{1}{\sqrt{1+t^2}}$. So, the corresponding probability density is given by: $$\eqalignno{f_Z(z)&={1\over2\...
user avatar
18 votes
Accepted

Integral of a Gaussian process

Question 1: Is $Y_t(\omega)$ well-defined? Answer: No, in general, $Y_t(\omega)$ is not well-defined; we need some additional assumption on the integrability of $X$ to ensure that $$\int_0^t |X_s(\...
user avatar
  • 113k
18 votes
Accepted

Why use 95% confidence interval?

From Wikipedia article 1.96 : The use of this number in applied statistics can be traced to the influence of Ronald Fisher's classic textbook, Statistical Methods for Research Workers, first ...
user avatar
  • 81.7k
17 votes
Accepted

P.d.f of the absolute value of a normally distributed variable

You can to take the cdf as a starting point. $P(|X| \leq x)=P(-x \leq X \leq x)$ $X\sim \mathcal N(0,1)$ $=P(X \leq x)-P(X \leq -x)$ $=P(X \leq x)-\left[ 1-P(X \leq x) \right]$ $=2 \cdot P(X \leq x)-1=...
user avatar
  • 28.3k
17 votes
Accepted

What is the purpose of $\frac{1}{\sigma \sqrt{2 \pi}}$ in $\frac{1}{\sigma \sqrt{2 \pi}}e^{\frac{(-(x - \mu ))^2}{2\sigma ^2}}$?

If you consider every possible outcome of some event you should expect the probability of it happening to be $1$, not $\sqrt{2\pi}$ so the constant scales the distribution to conform with the normal ...
user avatar
16 votes

How was the normal distribution derived?

The Normal distribution came about from approximations of the binomial distribution (de Moivre), from linear regression (Gauss), and from the central limit theorem. The derivation given by Tim relates ...
user avatar
16 votes
Accepted

Lower bound of Gaussian tail?

There is a standard lower estimate for $1-\Phi(x) = P(N(0,1)\ge x)$: for all $x>0$, $$ 1-\Phi(x)> \frac{x}{x^2+1}\varphi(x) = \frac{x}{x^2+1}\frac1{\sqrt{2\pi}}e^{-x^2/2}. $$ You can find the ...
user avatar
  • 23.7k
16 votes

Gaussian distribution is isotropic?

I'd just like to add a bit of visuals to the other answers. When the variables are independent, i.e. the distrubtion is isotropic, it means that the distribution is aligned with the axis. For ...
user avatar
16 votes
Accepted

Conditional expectation of a joint normal distribution

I've found an answer that I'm happy with: $$ Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X $$ is jointly normal with $X$ and uncorrelated, hence independent. Therefore $$\mathbb{E}[Y|X] = \...
user avatar
15 votes
Accepted

Normalized vector of Gaussian variables is uniformly distributed on the sphere

The two word answer is "polar coordinates". In more detail, let $f:S^{n-1}\to\Bbb R$ be a continuous function. Then $$ \eqalign{ \Bbb E[f(X)] &=\int_{\Bbb R^n}f(x_1/z,\ldots,x_n/z)(2\pi)^{-n/2}e^{...
user avatar
  • 22.9k
15 votes

Closed-form analytical solutions to Optimal Transport/Wasserstein distance

Although a bit old, this is indeed a good question. Here is my bit on the matter: Regarding Gaussian Mixture Models: A Wasserstein-type distance in the space of Gaussian Mixture Models, Julie Delon ...
user avatar
13 votes

Triangular vs Normal distribution

The best approximation in the $L^2$ sense is given by the value of $\alpha\in\mathbb{R}^+$ for which: $$ \frac{d}{d\alpha}\left(\frac{1}{2\pi}\int_{|x|\geq \alpha}e^{-x^2}\,dx + \int_{-\alpha}^{\...
user avatar
13 votes
Accepted

How is the entropy of the normal distribution derived?

Notice that $\ln(\color{blue}{\sqrt{\color{black}{x}}}) = \ln(x^{\color{blue}{\frac{1}{2}}}) = \color{blue}{\frac{1}{2}}\ln(x)$ and that $\ln(y) \color{red}{+} \ln(z) = \ln(y \color{red}{\cdot} z)$ ...
user avatar
13 votes
Accepted

Mean of squared $ L_2 $ norm of Gaussian random vector

The expectation of sum equals to the sum of expectations whenever they are exist: $$E\lvert \lvert X \rvert \rvert _2 ^2=E[X_1^2+\ldots+X_N^2]=E[X_1^2]+\ldots+E[X_N^2] = \text{Var}[X_1]+\ldots+\text{...
user avatar
  • 13.4k
12 votes

Product of Two Multivariate Gaussians Distributions

I depends on the information you have and the quantities you want to get out. If you have the covariance matrices themselves then you should use the formula $$ \Sigma_3 = \Sigma_1(\Sigma_1 + \Sigma_2)...
user avatar
  • 342
12 votes

Uniform distribution on the surface of unit sphere

Suppose $X=(X_1, X_2, \ldots, X_n)$ iid and $X_1 \sim N(0,1)$, then $X \sim N(0, I_n)$, where $N(0, I_n)$ is the multivariate normal distribution with zero-mean and identity covariance matrix. From ...
user avatar
  • 14.9k
12 votes

Proof of the affine property of normal distribution for a landscape matrix

One should approach this through characteristic functions. Recall that $X$ is normal $N(\mu,\Sigma)$ if and only if, for every deterministic vector $t$ of size $N\times1$, $$ E(\mathrm e^{\mathrm it'X}...
user avatar
  • 272k
12 votes

Why we consider log likelihood instead of Likelihood in Gaussian Distribution

First of all as stated, the log is monotonically increasing so maximizing likelihood is equivalent to maximizing log likelihood. Furthermore, one can make use of $\ln(ab) = \ln(a) + \ln(b)$. Many ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible