On the correct definition of convolution of probability density functions in polar coordinates Suppose $f$ and $g$ are two probability density functions on $\mathbb{R}^2$, and denote $F$ and $G$ to be the corresponding density functions written in polar coordinates. For instance, since we have $$1 = \int f(x_1,x_2) dx_1dx_2 = \int f(r\cos\theta,r\sin\theta) rdrd\theta,$$ we obtain $F(r,\theta) = f(r\cos\theta,r\sin\theta) r$ as the polar coordinate version of $f$. I am particularly interested in the correct way we should define convolution of $F$ and $G$. On the one hand, since $$(f*g)(y_1,y_2) = \int f(x_1,x_2)g(y_1-x_1,y_2-x_2)dx_1dx_2,$$ we expect
\begin{align*}
(F*G)(r,\theta) &= \int f(x_1,x_2)g(r\cos\theta-x_1,r\sin\theta-x_2)rdx_1dx_2 \\
&= \int f(\tilde{r}\cos\tilde{\theta},\tilde{r}\sin\tilde{\theta})g(r\cos\theta-\tilde{r}\cos\tilde{\theta},r\sin\theta-\tilde{r}\sin\tilde{\theta}) r\tilde{r}d\tilde{r}d\tilde{\theta}.
\end{align*}
While on the other hand, it is also reasonable to define $(F*G)(r,\theta)$ as $$(F*G)(r,\theta) = \int F(\tilde{r},\tilde{\theta})G(r-\tilde{r},\theta-\tilde{\theta}) d\tilde{r}d\tilde{\theta},$$ so that
$$(F*G)(r,\theta) = \int f(\tilde{r}\cos\tilde{\theta},\tilde{r}\sin\tilde{\theta})\tilde{r}\cdot g((r-\tilde{r})\cos(\theta-\tilde{\theta}),(r-\tilde{r})\sin(\theta-\tilde{\theta}))(r-\tilde{r}) d\tilde{r}\tilde{\theta}.$$ I am a bit confused as to which definition of the convolution of $F*G$ should I adopt... Thanks very much!
 A: Let me rewrite the two formulas you gave;
$$
f\ast g(y_1, y_2)=\iint_{\mathbb R^2} f(y_1-x_1, y_2-x_2)g(x_1, x_2)\, dx_1 dx_2, $$
and
$$
f\star g(r, \theta)=\int_{-\infty}^\infty\int_0^{2\pi} F(r-r', \theta-\theta')G(r', \theta')\, dr'd\theta',$$
(where $F(r, \theta), G(r, \theta)$ are related to $f, g$ as in your question).
The correct one, whatever that means, is the first.
First, the second formula has a problem, in that $F$ and $G$ need not be defined for negative $r$. We may solve this by prescribing that $F(-r,\theta)=-F(r, \theta)$, which is the correct behavior in the Gaussian case $F(r, \theta)=\tfrac{1}{\pi}re^{-r^2}$. In general, $f\ast g\ne f\star g$; for example, in the Gaussian case,
$$
f\ast g=\frac{1}{2\pi} e^{-\frac{x_1^2+x_2^2}{2}}, $$
while
$$
f\star g=\tfrac2\pi\int_{-\infty}^\infty (r-r')(r')e^{-r^2-2(r')^2-2rr'}\, dr'.$$
Now, the importance of $\ast$ is that, if $X, Y$ are independent random variables, and their densities are $f, g$ respectively, then $X+Y$ has density $f\ast g$. This is not the case for $\star$, as we have just seen. Thus, $\star$ is not a natural formula from the point of view of probability.
