88
votes
Accepted
Gaussian distribution is isotropic?
TLDR: An isotropic gaussian is one where the covariance matrix is represented by the simplified matrix $\Sigma = \sigma^{2}I$.
Some motivations:
Consider the traditional gaussian distribution:
$$
\...
- 996
66
votes
Accepted
Why we consider log likelihood instead of Likelihood in Gaussian Distribution
It is extremely useful for example when you want to calculate the
joint likelihood for a set of independent and identically distributed points. Assuming that you have your points:
$$X=\{x_1,x_2,\ldots,...
- 1,192
44
votes
Accepted
A Mathematical Paradox About Probabilities
Something to think about:
Since the coin flips are independent, and assuming the coin is fair, the probability that ten coin flips land heads is:
$$P(H)\cdot P(H)\cdot P(H)\cdot P(H)\cdot P(H)\cdot ...
- 6,337
25
votes
Accepted
Basic Calc - why can't I solve this integral?
When you take the derivative of $$\frac{1}{2x}e^{x^{2}}$$ you need to use the product and chain rules. You get
$$\frac{d}{dx}\frac{1}{2x}e^{x^{2}} = -\frac{1}{2x^{2}}e^{x^{2}}+\frac{1}{2x}2xe^{x^{2}} ...
- 9,159
23
votes
Product of Two Multivariate Gaussians Distributions
An alternative expression of the PDF proportional to the product is:
$\Sigma_3 = \Sigma_1(\Sigma_1 + \Sigma_2)^{-1}\Sigma_2$
$\mu_3 = \Sigma_2(\Sigma_1 + \Sigma_2)^{-1}\mu_1 + \Sigma_1(\Sigma_1 + \...
- 331
21
votes
Scaling the normal distribution?
On the first page of the cited document, $X_1$ and $X_2$ were previously defined to be two (distinct) independent, identically distributed random variables. For your purposes, the "identically ...
- 86.6k
19
votes
A Mathematical Paradox About Probabilities
All the answers provided explain why there is no "paradox." I would like to provide you an answer that is rather intuitive (hopefully) than formal.
Your question is a good example of what is known as ...
- 1,157
18
votes
Is the product of two Gaussian random variables also a Gaussian?
As pointed out by Davide Giraudo, the characteristic function of Z is
$\varphi_Z (t) = \frac{1}{\sqrt{1+t^2}}$. So, the corresponding probability density is given by:
$$\eqalignno{f_Z(z)&={1\over2\...
18
votes
Accepted
Integral of a Gaussian process
Question 1: Is $Y_t(\omega)$ well-defined?
Answer: No, in general, $Y_t(\omega)$ is not well-defined; we need some additional assumption on the integrability of $X$ to ensure that $$\int_0^t |X_s(\...
- 113k
18
votes
Accepted
Why use 95% confidence interval?
From Wikipedia article 1.96 :
The use of this number in applied statistics can be traced to the
influence of Ronald Fisher's classic textbook, Statistical Methods for
Research Workers, first ...
- 81.7k
17
votes
Accepted
P.d.f of the absolute value of a normally distributed variable
You can to take the cdf as a starting point.
$P(|X| \leq x)=P(-x \leq X \leq x)$
$X\sim \mathcal N(0,1)$
$=P(X \leq x)-P(X \leq -x)$
$=P(X \leq x)-\left[ 1-P(X \leq x) \right]$
$=2 \cdot P(X \leq x)-1=...
- 28.3k
17
votes
Accepted
What is the purpose of $\frac{1}{\sigma \sqrt{2 \pi}}$ in $\frac{1}{\sigma \sqrt{2 \pi}}e^{\frac{(-(x - \mu ))^2}{2\sigma ^2}}$?
If you consider every possible outcome of some event you should expect the probability of it happening to be $1$, not $\sqrt{2\pi}$ so the constant scales the distribution to conform with the normal ...
- 7,713
16
votes
How was the normal distribution derived?
The Normal distribution came about from approximations of the binomial distribution (de Moivre), from linear regression (Gauss), and from the central limit theorem. The derivation given by Tim relates ...
- 491
16
votes
Accepted
Lower bound of Gaussian tail?
There is a standard lower estimate for $1-\Phi(x) = P(N(0,1)\ge x)$: for all $x>0$,
$$
1-\Phi(x)> \frac{x}{x^2+1}\varphi(x) = \frac{x}{x^2+1}\frac1{\sqrt{2\pi}}e^{-x^2/2}.
$$
You can find the ...
- 23.7k
16
votes
Gaussian distribution is isotropic?
I'd just like to add a bit of visuals to the other answers.
When the variables are independent, i.e. the distrubtion is isotropic, it means that the distribution is aligned with the axis.
For ...
- 279
16
votes
Accepted
Conditional expectation of a joint normal distribution
I've found an answer that I'm happy with:
$$ Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X $$ is jointly normal with $X$ and uncorrelated, hence independent.
Therefore
$$\mathbb{E}[Y|X] = \...
- 585
15
votes
Accepted
Normalized vector of Gaussian variables is uniformly distributed on the sphere
The two word answer is "polar coordinates".
In more detail, let $f:S^{n-1}\to\Bbb R$ be a continuous function. Then
$$
\eqalign{
\Bbb E[f(X)]
&=\int_{\Bbb R^n}f(x_1/z,\ldots,x_n/z)(2\pi)^{-n/2}e^{...
- 22.9k
15
votes
Closed-form analytical solutions to Optimal Transport/Wasserstein distance
Although a bit old, this is indeed a good question. Here is my bit on the matter:
Regarding Gaussian Mixture Models:
A Wasserstein-type distance in the space of Gaussian Mixture Models, Julie Delon
...
- 575
13
votes
Triangular vs Normal distribution
The best approximation in the $L^2$ sense is given by the value of $\alpha\in\mathbb{R}^+$ for which:
$$ \frac{d}{d\alpha}\left(\frac{1}{2\pi}\int_{|x|\geq \alpha}e^{-x^2}\,dx + \int_{-\alpha}^{\...
- 344k
13
votes
Accepted
How is the entropy of the normal distribution derived?
Notice that $\ln(\color{blue}{\sqrt{\color{black}{x}}}) = \ln(x^{\color{blue}{\frac{1}{2}}}) = \color{blue}{\frac{1}{2}}\ln(x)$ and that $\ln(y) \color{red}{+} \ln(z) = \ln(y \color{red}{\cdot} z)$ ...
- 3,621
13
votes
Accepted
Mean of squared $ L_2 $ norm of Gaussian random vector
The expectation of sum equals to the sum of expectations whenever they are exist:
$$E\lvert \lvert X \rvert \rvert _2 ^2=E[X_1^2+\ldots+X_N^2]=E[X_1^2]+\ldots+E[X_N^2] = \text{Var}[X_1]+\ldots+\text{...
- 13.4k
12
votes
Product of Two Multivariate Gaussians Distributions
I depends on the information you have and the quantities you want to get out.
If you have the covariance matrices themselves then you should use the formula
$$
\Sigma_3 = \Sigma_1(\Sigma_1 + \Sigma_2)...
- 342
12
votes
Uniform distribution on the surface of unit sphere
Suppose $X=(X_1, X_2, \ldots, X_n)$ iid and $X_1 \sim N(0,1)$, then $X \sim N(0, I_n)$, where $N(0, I_n)$ is the multivariate normal distribution with zero-mean and identity covariance matrix. From ...
- 14.9k
12
votes
Proof of the affine property of normal distribution for a landscape matrix
One should approach this through characteristic functions. Recall that $X$ is normal $N(\mu,\Sigma)$ if and only if, for every deterministic vector $t$ of size $N\times1$,
$$
E(\mathrm e^{\mathrm it'X}...
- 272k
12
votes
Why we consider log likelihood instead of Likelihood in Gaussian Distribution
First of all as stated, the log is monotonically increasing so maximizing likelihood is equivalent to maximizing log likelihood. Furthermore, one can make use of $\ln(ab) = \ln(a) + \ln(b)$. Many ...
- 1,401
Only top scored, non community-wiki answers of a minimum length are eligible