Does the Integral over the product of distributions result in another distribution?

Question

In this post one guy asks why those likelihoods in Maximum Likelihood get multiplied.

It's clear why they get multiplied. I have another problem. In my textbook it is state that for each sample the same distribution is assumed. Those likelihoods than get multiplied which results in $p(D|\theta)$:

So its quite good illustrated that new distribution will have its mean, where all the samples underlying distribution has most probably its mean.

Now its peak has to be quite low since e.g. the point far to the right and its distribution are quite small at the actual mean. However, is the resulting $p(D|\theta)$ not a real distribution? I somehow assumed the integral over it should result in 1. Doesn't seem so when looking at the peak at about 1.5e-7, but maybe it does because the sides go to minus and plus infinity. Tested it for two distributions in Wolfram Alpha (variance is 1):

$$\int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}}e^{-\frac{(x-1)^2}{2}} \cdot \frac{1}{\sqrt{2\pi}}e^{-\frac{(x-4)^2}{2}}dx = 0.0297326$$

Different means resulted in different values. Had to look the calculus up but I guess this integral even goes to 0 if the distance between to means gets bigger. Still if you would zoom in you should get something bell curve like with a really small amplitude... Maybe I'm just mixing up distributions, likelihood and probability.

Answer 1

You’re confusing a few concepts here.

1. If you want to take the product of densities, remember that the variables are different! By independence the likelihood would be $f_1(x_1)f_2(x_2)$ where each $f$ denotes the density of each random variable. This is a density on $\mathbb{R}^2$.

2. If you want the distribution of the likelihood, you need to transform the joint distribution of the variables to the product of all of them. Letting $X_1$ and $X_2$ denote the random variables corresponding to your observations. You want the density of $X_1X_2$ but this is NOT the product of the densities.

Answer 2

Found the answer in the book. As it is assumed the samples are i.i.d. So the product equals the joint probability. Calculating this product with respect to the parameter theta (the mean) results in the middle graph. This is indeed not a probability distribution. It's just used to see it's maximum.

Sorry for the confusion.