Why is chi-square distribution with 2 degrees of freedom an exponential distribution? Is there any explanation on why these two distributions are equivalent?
How can the sum of two square of Gaussians represents the limit of a geometric distribution?
I found an answer here, which says it is just coincidence. But I have a hard time believing it.
 A: There's an "interesting" mathematical fact here.  It's involved in the usual technique for evaluating $\int_{-\infty}^\infty \mathrm{e}^{-x^2} \,\mathrm{d}x$.  Let $I = \int_{-\infty}^\infty \mathrm{e}^{-x^2} \,\mathrm{d}x$ and compute:  \begin{align*}
    I^2 &= \int_{-\infty}^\infty \mathrm{e}^{-x^2} \,\mathrm{d}x \int_{-\infty}^\infty \mathrm{e}^{-y^2} \,\mathrm{d}y  \\
      &= \int_{-\infty}^\infty \int_{-\infty}^\infty \mathrm{e}^{-(x^2+y^2)} \,\mathrm{d}x \,\mathrm{d}y  \\
      &= \int_{-\pi}^\pi \int_{0}^\infty \mathrm{e}^{-r^2} r\,\mathrm{d}r \,\mathrm{d}\theta  \text{,}
\end{align*} which we can complete by the substitution $u = r^2, \mathrm{d}u = 2 r \,\mathrm{d}r$.  When we do this, the integral becomes (skipping extraneous details) $\int \mathrm{e}^{-u} \,\mathrm{d}u$, as you have observed.  (Note that the circles of constant $r$ correspond to $(x,y)$ pairs whose square sum to the same value, so we would integrate this in the $\theta$ direction to get the PDF of the sum of squares of two normal variables.)
Note that this doesn't work for the sum of three variables.  The differential element in spherical coordinates is $r^2 \sin \theta \,\mathrm{d}r\,\mathrm{d}\theta\,\mathrm{d}\phi$, but we need the first power of $r$ to make the substitution work.  
In this sense, this is just a happy coincidence -- the 2-dimensional problem allows us to conjure up the factor of $r$ needed to evaluate the integral.  In another sense it is not a coincidence -- the $x^2$ in the exponent and that we can take products of independent PDFs ensures that this is going to work, that we will get the squared distance from the origin to $(x,y)$ in the exponent, precisely the combination of variables we want to know about.
