Product of two Gaussian PDFs is a Gaussian PDF, but Product of two Gaussian Variables is not Gaussian

Question

The Product of Two Gaussian Random Variables is not Gaussian distributed:

• Is the product of two Gaussian random variables also a Gaussian?
• Also Wolfram Mathworld
• So this is saying $X \sim N(\mu_1, \sigma_1^2)$, $Y \sim N(\mu_2, \sigma_2^2)$ then $XY \sim W$ where W is some other distribution, that is not Gaussian

But the product of two Gaussian PDFs is a Gaussian PDF:

• Calculate the product of two Gaussian PDF's
• Full Proof
• This tutorial which I am trying to understand Writes: $N(\mu_1, \sigma_1^2)\times N(\mu_2, \sigma_2^2) = N(\frac{\sigma_1^2 \mu_2 + \sigma_2^2 \mu_1}{\sigma_1^2 + \sigma_2^2},\frac{1}{\frac{1}{\sigma_1^2} + \frac{1}{\sigma_2^2}})$

What is going on here?

What am I doing when I take the product of two pdfs vs. when I take the product of two variables from the pdfs?

When (what physical situation) is described by one, and what by the other? (I think a few real world examples would clear things up for me)

Answer 1

The product of the PDFs of two random variables $X$ and $Y$ will give the joint distribution of the vector-valued random variable $(X,Y)$ in the case that $X$ and $Y$ are independent. Therefore, if $X$ and $Y$ are normally distributed independent random variables, the product of their PDFs is bivariate normal with zero correlation.

On the other hand, even in the case that $X$ and $Y$ are IID standard normal random variables, their product is not itself normal, as the links you provide show. The product of $X$ and $Y$ is a scalar-valued random variable, not a vector-valued one as in the above case.

Answer 2

1.) The first example is already sufficient. Just to throw in another one for a sum of Gaussian variables, consider diffusion: at each step in time a particle is perturbed by a random, Gaussian-distributed step in space. At each time the distribution of its possible positions in space will be a Gaussian because the total displacement is the sum of a bunch of Gaussian-distributed displacements, and the sum of Gaussian variables is Gaussian.

2.) The second situation (product of Gaussian PDFs) is confusing because the resulting function is a Gaussian, but it is not a probability distribution because its not normalized! Nevertheless, there are physical situations in which the product of two Gaussian PDFs is useful. See below.

TL;DR - a physical example for a product of Gaussian PDFs comes from Bayesian probability. If our prior knowledge of a value is Gaussian, and we take a measurement which is corrupted by Gaussian noise, then the posterior distribution, which is proportional to the prior and the measurement distributions, is also Gaussian.

For example:

Suppose you are trying to measure a constant, unknown, value $X$. You can take measurements of it, with Gaussian noise, your measurement model is $\tilde{X} = X + \epsilon$. Finally, suppose you have a Gaussian prior distribution for $X$. Then, the posterior distribution after taking a measurement is

$$P[X\mid \tilde{X}] = \frac{P[\tilde{X}\mid X] P[X]}{P[\tilde{X}]}$$

As is fashionable in beysian probability, we throw out the value $P[\tilde{X}]$, because it doesn't depend explicitly on $X$, so we can ignore it for now and normalize later.

Now, our assumption is that the prior, $P[X]$, is Gaussian. The measurement model tells us that $P[\tilde{X}\mid X]$ is Gaussian, in particular $P[\tilde{X}\mid X] = N[\Sigma_{\epsilon},X]$. Since the product of two Gaussians is a Gaussian, the posterior probability is Gaussian. It is not normalized, but that is where $P[\tilde{X}]$ (which we "threw out" earlier) comes in. It must be exactly the right value to normalize this distribution, which we can now read off from the variance of the Gaussian posterior.

What you should really take away from this is that Gaussians are magical [1]. I don't know of any other PDF which has this property. This is why, for example, Kalman filters work so darn well. Kalman filters utilize both of these properties, and that is how you get a super-efficient algorithm for state estimation for a linear dynamical system with Gaussian noise.

[1] - Gaussians are not actually magical, but perhaps they are mathemagical.

Answer 3

In brief, you were confused by two totally different concepts.

1. For the first, you are calculating the distribution of transformed random variables. here, you specify that as the product of XY.

2. For the second, you just calculate the product of two functions $\phi(x)\phi(y)$, which happen to be the PDF of two normal random variables.

For some details, check here. In general, if you want to calculate the PDF of XY, you need to figure out $F(t) = P(XY < t)$, then $f(t) = F'(t)$. As for the product of two functions, that's easy as you can see.

Answer 4

Intuition 1 (Multiplying random variables): Suppose kids save money Y for X days before giving up. X is random iid normally distributed variable from 0 days to 10 days. Average kid saves for 5 days. Average kid saves 0.30 a day (mostly between .10 and .50). What's the distribution?

Now are the savings of a kid normally distributed? We know the mean, median, mode of a normal distribution are same as it is symmetric with a standard deviation.

The average savings are clearly $0.30 * 5 = 1.50. The max savings are 5 and the min is 0. The median of 5 and 0 is 2.50. The mean and median don't match so the product is not normally distributed. The distribution is shifted to the left from the 2.50 mark. The probability that a kid saved 1 is higher than the probability that he saved 4.

Intuition 2 (Multiplying Gaussian PDFs): Now you're multiplying not the numbers but the functions together. The multiplying is just a bunch of algebra and the resulting function also fits the form factor of a Gaussian. The proof for that is given in your link. It means if you have populations of kids there will be a Gaussian representing their savings with some average savings from each sub population. Basically, a mixture of gaussians is a gaussian. See https://en.m.wikipedia.org/wiki/Mixture_model for how various distributions mix. It’s useful for building machine learning models. Maybe given distributions of daily savings and total savings you want to establish the distribution of how long the kids tend to save.

Answer 5

I am surprised @heropup's answer is accepted, because the original question is about the product of two Gaussian PDFs resulting in a new, scalar PDF (instead of a vector joint PDF like @heropup supposes). Although we have two Gaussian PDFs of different variables, i.e.

$$ f_1(x) \sim N(\mu_1,\sigma_1^2)\\ f_2(y) \sim N(\mu_2,\sigma_2^2), $$

they are multiplied by assuming some dummy variable $t$:

$$ f = f_1(t) f_2(t) \sim N(\hat{\mu},\hat{\sigma}^2) $$

which is a scalar PDF and not a joint PDF! An example is given below, with $\mu_1=5$, $\mu_2=10$ and $\sigma_1=\sigma_2=1$. The resulting $f$ is normalized so that it is a valid PDF. Due to the exponential nature of a Gaussian PDF, it peaks where both $f_1$ and $f_2$ are large.

The resulting $f$ should not be interpreted as the PDF of $Z=X\cdot Y$! This is obvious as the peak should not be near $7.5$ but rather near $\mu_1 \mu_2=50$ (see plot below).

Note that $f$ in this plot is generally not Gaussian.