This is something I am struggling to understand. I saw other proofs about this, but I want to try and do this myself using convolutions.
I always thought that the sums of two independent Gaussian Random Variables are always Gaussian. I tried to prove this using convolutions:
For two random variables $X$ and $Y$, define a new random variable $Z = X+Y$. Then, the distribution of this new random variable $Z$ can be given by the convolution :
$$ r(z) = (p * q)(z) = \int_{-\infty}^{\infty} p(x) q(z - x) dx $$
If $X$ and $Y$ are both independent Gaussians, I heard that their sum is also Gaussian. I tried to prove this myself:
$$ (f_X*f_Y)(z) = \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi\sigma_1^2}} e^{ -\frac{(x-\mu_1)^2}{2\sigma_1^2} } \frac{1}{\sqrt{2\pi\sigma_2^2}} e^{ -\frac{(z-x-\mu_2)^2}{2\sigma_2^2} } dx $$
$$ = \frac{1}{2\pi\sigma_1\sigma_2} \int_{-\infty}^{\infty} e^{ -\frac{(x-\mu_1)^2}{2\sigma_1^2} -\frac{(z-x-\mu_2)^2}{2\sigma_2^2} } dx $$
$$ = \frac{1}{2\pi\sigma_1\sigma_2} \int_{-\infty}^{\infty} e^{ -\frac{1}{2} \left( \frac{(x-\mu_1)^2}{\sigma_1^2} + \frac{(z-x-\mu_2)^2}{\sigma_2^2} \right) } dx $$
From here the steps/manipulation get a bit confusing for me, but I tried to continue:
$$ = \frac{1}{2\pi\sigma_1\sigma_2} \int_{-\infty}^{\infty} e^{ -\frac{1}{2} \left( \frac{x^2}{\sigma_1^2} - 2\frac{x\mu_1}{\sigma_1^2} + \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} - 2\frac{zx}{\sigma_2^2} + \frac{x^2}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2} \right) } dx $$
$$ = \frac{1}{2\pi\sigma_1\sigma_2} \int_{-\infty}^{\infty} e^{ -\frac{1}{2} \left( \frac{x^2}{\sigma_1^2} + \frac{x^2}{\sigma_2^2} - 2\frac{x\mu_1}{\sigma_1^2} + \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} - 2\frac{zx}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2} \right) } dx $$
$$ = \frac{1}{2\pi\sigma_1\sigma_2} \int_{-\infty}^{\infty} e^{ -\frac{1}{2} \left( x^2\left(\frac{1}{\sigma_1^2} + \frac{1}{\sigma_2^2}\right) - 2x\left(\frac{\mu_1}{\sigma_1^2} + \frac{z}{\sigma_2^2}\right) + \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2} \right) } dx $$
I think we can "complete the square". The term inside the exponent can be rewritten as:
$$ -\frac{1}{2} \left( x^2\left(\frac{1}{\sigma_1^2} + \frac{1}{\sigma_2^2}\right) - 2x\left(\frac{\mu_1}{\sigma_1^2} + \frac{z}{\sigma_2^2}\right) + \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2} \right) $$
This is a quadratic expression in the form $ax^2 + bx + c$. To complete the square, we want to rewrite this in the form $a(x-h)^2 + k$. By choosing the following values we can complete the square:
$$a = \frac{1}{\sigma_1^2} + \frac{1}{\sigma_2^2}$$ $$b = -\left(\frac{\mu_1}{\sigma_1^2} + \frac{z}{\sigma_2^2}\right)$$. $$h = \frac{\mu_1/\sigma_1^2 + z/\sigma_2^2}{1/\sigma_1^2 + 1/\sigma_2^2}$$.
If we substitute these back into the previous equation, we get:
$$a(x-h)^2 = \left(\frac{1}{\sigma_1^2} + \frac{1}{\sigma_2^2}\right)\left(x - \frac{\mu_1/\sigma_1^2 + z/\sigma_2^2}{1/\sigma_1^2 + 1/\sigma_2^2}\right)^2$$
The remaining terms in the exponent can be grouped together to form $$k = \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2} - h^2$$.
Thus in total:
$$a(x-h)^2 + k = \left(\frac{1}{\sigma_1^2} + \frac{1}{\sigma_2^2}\right)\left(x - \frac{\mu_1/\sigma_1^2 + z/\sigma_2^2}{1/\sigma_1^2 + 1/\sigma_2^2}\right)^ + \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2} - h^2$$
Now, if we revisit the original convolution formula and make these substitutions:
$$ (f_X*f_Y)(z) = \frac{1}{2\pi\sigma_1\sigma_2} \int_{-\infty}^{\infty} e^{ -\frac{1}{2} \left( a(x-h)^2 + k \right) } dx $$
$$ (f_X*f_Y)(z) = \frac{1}{2\pi\sigma_1\sigma_2} e^{-\frac{1}{2} \left( \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2} - \left(\frac{\mu_1/\sigma_1^2 + z/\sigma_2^2}{1/\sigma_1^2 + 1/\sigma_2^2}\right)^2 \right)} $$
Using the formula $ax^2 + bx + c$, the term inside the exponent can be further simplified
$$ -\frac{1}{2} \left( \frac{x^2}{\sigma_1^2} + \frac{x^2}{\sigma_2^2} - 2\frac{x\mu_1}{\sigma_1^2} + \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} - 2\frac{zx}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2} \right) $$
$$ -\frac{1}{2} \left( x^2\left(\frac{1}{\sigma_1^2} \frac{1}{\sigma_2^2}\right) - 2x \left(\frac{\mu_1}{\sigma_1^2} + \frac{z}{\sigma_2^2}\right) + \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2} \right) $$
Here, $a = \frac{1}{\sigma_1^2} + \frac{1}{\sigma_2^2}$, $b = -\left(\frac{\mu_1}{\sigma_1^2} + \frac{z}{\sigma_2^2}\right)$, and $c = \frac{\mu_1^2}{\sigma_1^2} + \frac{z^2}{\sigma_2^2} + \frac{\mu_2^2}{\sigma_2^2}$.
I think I am really lost now, I am not sure how to justify that the above can be used to show that the resulting convolution is Gaussian with $\mu = \mu_1 + \mu_2$ and $\sigma^2 = \sigma^2_1 + \sigma^2_2$.
Can someone please help me out with this and continue the proof? Ideally, I would like to look at the exponent term and manipulate it until it becomes of the form: $\frac{x-\mu}{\sigma}$, i.e.
$$\frac{z - (\mu_1 + \mu_2)}{\sigma^2_1 + \sigma^2_2}$$
Then I could right away see that: $\mu = \mu_1 + \mu_2$ and $\sigma^2 = \sigma^2_1 + \sigma^2_2$.
Thank you!
