Fourier transform with a different sign convention I am working with a paper where the authors make heavy use of Fourier transformations - of functions and finite Borel measures.
I guess to be consistent with the probabilistic characteristic function they introduced the Fourier transform (on measures) with the following sign convention
$$
\mathcal F[\mu](u)=\int e^{iux}\mu(dx)
$$
they never stated the Fourier transform on function spaces like $L^1(\mathbb R)$ but still use it. Anyway, to be consistent in what I do I thought I keep their notation and also introduce the Fourier transform on $L^1(\mathbb R)$ etc. as
$$
\mathcal F[f](u)=\int e^{iux}f(x)dx \text{ and }\mathcal F^{-1}[f](u)=\frac {1}{2\pi}\int e^{-iux}f(x)dx 
$$ 
so pretty much standard as in Folland and alike except for the different sign convention in the exponential.
Does anyone know a reference where such a convention is used? Just to be sure, it really is just a convention, right? It just reverses the way of integration but that's about it and we loose the unitary property, but that's due to the leading factor - if I have missed something here just let me know.
 A: After continuing the search I found a beautiful summary of the notational hazard of the Fourier transform, it's taken from The Fourier transform and its application, lecture notes by Brad Osgood on page 74/75, it reads:

Following the helpful summary provided by T. W. Körner in his book
  Fourier Analysis
  , I will summarize
  the many irritating variations. To be general, let’s write:
  $$
\mathcal Ff(s)=\frac 1 A\int_{-\infty}^{\infty}e^{iBst}f(t)\;dt
$$
  The choices that are found in practice are
  $$
\begin{array}{c c}
 A=\sqrt {2\pi}&B=\pm1 \\ 
 A=1&B=\pm2\pi \\ 
 A=1&B=\pm1  
\end{array}
$$
  Happy hunting and good luck.

Though this is still no reference where it actually is introduced as in the question (so $ A=1,B=+1$) but it's something (and something I can refer to).
A: The sign convention used for Fourier transforms (either plus or minus) is simply that; a convention, an agreement of terms. It is not that one is better than the other. Early mathematicians (and also many physicists) chose a negative sign for the “inverse” transformation from the frequency domain into the time domain, after which most other mathematicians followed suit. Yet engineers begged to differ, and in their disciplines they stuck to the alternative convention: a positive sign for the inverse transform. The problem lies not in that one is better than the other. The big problem arises when one group talks to the other, or reads each other’s papers; then all sorts of disagreements, consequential errors and confusions may arise.
At first sight it might be thought that “the sign convention does not really matter”, yet beside the superficial change in the sign of the transform, profound theoretical consequences follow. If one is aware of these, then no problem, but in most cases the underlying details remain overlooked or ignored, in which case serious errors could occur. For one, with the change in sign all frequency response functions (or transfer functions) get complex-conjugated, as a result of which the poles of these functions move from the upper to the lower complex half-plane, or vice-versa. That also changes the appropriate contours over which integrals can be carried out (i.e. where they remain bounded), and also affects causality as well as energy conservation & dissipation. As long as one sticks to one of these conventions without ever mixing, all remains well.
Consider the following example of a mass-damper-spring system subjected to some causal, external force of finite duration. This is the same as the equation for a capacitor-resistor-inductor electrical circuit subjected to a time varying electrical source. The equation of motion of the mechanical oscillator is
$$m\,\ddot u + c\dot u + ku = f\left( t \right)$$
where superscripted dots represent differentiation with respect to time, and the system starts at rest. Using the engineering convention, a forward transform from the time domain into the frequency can be written as
$$\int\limits_{ - \infty }^\infty  {\left( {m\,\ddot u + c\dot u + ku} \right){{\rm{e}}^{ - {\rm{i}}\omega t}}dt}  = \int\limits_{ - \infty }^\infty  {f\left( t \right){{\rm{e}}^{ - {\rm{i}}\omega t}}dt} 
.$$
After applying integration by parts on the left-hand side which make use of the starting conditions, we obtain
$$\left( { - {\omega ^2}m + {\rm{i}}\omega c + k} \right)U = F$$
so the frequency response function is
$$U = \frac{F}{{k - {\omega ^2}m + {\rm{i}}\omega c}}$$
This function has poles at
$$k - {\omega ^2}m + {\rm{i}}\omega c = 0 $$
which yields the roots
$${\omega _j} = \frac{{{\rm{i}}\,c \pm \sqrt {4mk - {c^2}} }}{{2m}} = {\omega _n}\left( {{\rm{i}}\,\zeta  \pm \sqrt {1 - {\zeta ^2}} } \right)$$
where  ${\omega _n} = \sqrt {k/m} $ is the natural frequency, and $\zeta  = {\textstyle{1 \over 2}}c/\sqrt {km} $ is the fraction of critical damping. Clearly, both of these lie in the upper complex half-plane. The fact that the lower complex half-plane has no poles indicates that the response is zero for negative times, that is, the oscillator satisfies causality.
Now, if this same problem had been studied by a mathematician, s/he would have arrived instead at a transfer function
$$U = \frac{F}{{k - {\omega ^2}m - {\rm{i}}\omega c}}$$
with a negative sign in the imaginary term. You will never, ever find that in any book on mechanical vibration. But it is often found in works on wave propagation, where mathematicians use their rule and engineers use the other rule. Since most of those works deal with purely elastic media, there are no damping terms, and therein lies the big danger: the transfer functions with one and the other convention look similar, but ultimately they are not. Problems of wave propagation are further encumbered by the need to consider two-dimensional space-time transforms, which add further obscurity to the conventions. Thus, consistency is the key.
A: The Fourier transform can more generally be defined as
$$F(\omega)=\mathcal{F}_t[f(t)](\omega )=\sqrt{\frac{|b|}{(2 \pi)^{1-a}}} \int\limits_{-\infty}^{\infty} f(t)\, e^{i b \omega t}\,dt\tag{1}$$
and the inverse Fourier transform as
$$f(t)=\mathcal{F}_{\omega }^{-1}[F(\omega)](t)=\sqrt{\frac{|b|}{(2 \pi)^{a+1}}} \int\limits_{-\infty}^{\infty } F(\omega)\, e^{-i b t \omega}\,d\omega\tag{2}$$
using two arbitrary constants $a$ and $b$ (see formulas (15) and (16) at Fourier Transform - from Wolfram MathWorld).
