In addition to other useful remarks, it might be worthwhile to note that thinking in terms of representations of a topological group $G$ (on topological vector spaces $V$) shows what "convolution" must be, in the following way. For simplicity, suppose $G$ is unimodular, in the sense that left and right Haar measures are the same.
Let $G\times V\to V$ be a continuous group respresentation, so including associativity $g(hv)=(gh)v$ and that the identity of $G$ acts trivially. For a broad class of topological vector spaces $V$ (quasi-complete locally convex, including Hilbert, Banach, Frechet, LF, their weak duals...) compactly-supported continuous functions $f$ on $G$ act by integrating (e.g., Gelfand-Pettis "weak" integrals suffice)
$$
f\cdot v \;=\; \int_G f(g)\;gv\;dg
$$
If we characterize an operation $f*F$ by requiring $(f*f)v=f(Fv)$, we are led to
to an expression (or two) for that convolution: first,
$$
f(Fv) \;=\; \int_G f(g)\;g (Fv)\;dg \;=\; \int_G f(g)\,g\Big(\int_G F(h)v\;dh\Big)dg
\;=\; \int\int f(g)\,F(h)\,ghv\;dh\,dg
$$
There are at least two reasonable choices now: replace $g$ by $gh^{-1}$, or replace $h$ by $g^{-1}h$. In the former, we have
$$
f(Fv) \;=\; \int\int f(gh^{-1})\,F(h)\;gv\;dh\,dg
\;=\; \int\Big(\int f(gh^{-1})\,F(h)\,dh\Big)\,gv\;dg
\;=\; \Big(g\to \int f(gh^{-1})\,F(h)\,dg\Big)\cdot v
$$
which shows that
$$
(f*F)(g)\;=\; \int_G f(gh^{-1})\,F(h)\;dh
$$
For the real numbers, this gives the $x-y$ rather than $x+y$. But, to my mind, a larger point is that we can deduce what convolution is, rather than "guessing" a "definition" and "checking" whether or not it works as we hope.