What is the square root of a Fourier transform? Given the Fourier transform defined like
$$
\mathcal F[f](\omega) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^\infty f(t) e^{-iwt} \mathrm{d}t
$$
how could one define a square root $\mathcal G$ of the operator $\mathcal F$ so that 
$$
\mathcal G^2[f] = F[f]
$$
My idea is to literally take the square root of $\mathcal F$ like
$$
\mathcal G[f](\omega) = \sqrt{\mathcal F[f](\omega)} = \sqrt{\frac{1}{\sqrt{2 \pi}} \int_{-\infty}^\infty f(t) e^{-iwt} \mathrm{d}t} = \frac{1}{\sqrt{\sqrt{2 \pi}}} \int_{-\infty}^\infty \sqrt{f(t)} e^{-\frac{iwt}{2}} \mathrm{d}t
$$
Then the square of $\mathcal G[f]$ would be
$$
\mathcal G^2[f](\omega) = \frac{1}{\sqrt{\sqrt{2 \pi}}} \frac{1}{\sqrt{\sqrt{2 \pi}}} \int_{-\infty}^\infty \int_{-\infty}^\infty \sqrt{f(t)} \sqrt{f(t')} e^{-\frac{iwt}{2}} e^{-\frac{iw't'}{2}} dt' dt
$$
But this doesn't quite equate to $\mathcal F[f]$ again. So I guess my approach of literally taking the square root is a bit too naive. How then could one define the square root of a Fourier transform?
 A: @Demophilus shows probably the most straight forward way to do it, but you can also add for example a shuffle (permutation) of the frequency components. If we assume the canonical decomposition by Demophilus is ${\bf SDS}^{-1}$ You can then bake it in the canonical transformation like this:
$${\bf SDPS}^{-1}$$
Where $\bf D$ is the diagonal matrix with the modified eigenvalues, for example like @Demophilus proposed in his answer. Furthermore, will need ${\bf P}^2 = \bf I$. And of course $$({\bf SDPS}^{-1})^2 = \bf F$$
The Fourier transform matrix.
To show that such a $\bf P$ exists, here is one for a 6 point FFT
$${\bf P} = \left[\begin{array}{cccccc}0&0&0&1&0&0\\0&0&0&0&0&1\\0&0&1&0&0&0\\1&0&0&0&0&0\\0&0&0&0&1&0\\
0&1&0&0&0&0\end{array}\right]$$
This one was found by ocular inspection on the eigenvalues. But we know results about the eigenvalues to the Fourier transform in general which pave the way to construct this in a more general setting.
A: The Fourier transform can be diagonalised. Let $h_n$ be the Hermite functions, then we have
$$
\mathcal F[h_n] = (-i)^n h_n.
$$
So a good definition for a square root of the Fourier transform would be
$$
\mathcal G[f] = \sum_{n=0}^\infty \langle f, h_n \rangle \mathcal G[h_n ] = \sum_{n=0}^\infty \langle f, h_n \rangle (-i)^{n/2} h_n.
$$
A closed form expression can be found in the other answers.
A: You have an operator $\mathcal{F}$ that is unitary and satisfies $\mathcal{F}^4=1$. Let $r=e^{2\pi i/4}=i$. Define polynomials
$$
  p_k(z) = \frac{\prod_{l=0,l\ne k}^{3}(z-r^l)}{\prod_{l=0,l\ne k}^3(r^k-r^l)}.
$$
Then $p_k(r^l)=\delta_{k,l}$ for $0 \le k,l \le 3$. Therefore,
$$
          p_1+p_2+p_3+p_4 = 1.
$$
(This is because this third order polynomial sum is $1$ at the four points $r,r^2,r^3,r^4$.) Likewise,
$$
         p_1(\mathcal{F})+p_2(\mathcal{F})+p_3(\mathcal{F})+p_4(\mathcal{F})=I, \\
          p_k(\mathcal{F})p_l(\mathcal{F})=0,\;\; k\ne l.
\\
        \mathcal{F}p_k(\mathcal{F})=r^{k-1}p_k(\mathcal{F}).
$$
Consequently,
$$
          \mathcal{F} = p_1(\mathcal{F})+rp_2(\mathcal{F})+r^2p_3(\mathcal{F})+r^3p_3(\mathcal{F}).
$$
Therefore,
$$
            S = p_1(\mathcal{F})+r^{1/2}p_2(\mathcal{F})+rp_3(\mathcal{F})+r^{3/2}p_4(\mathcal{F})
$$
satisfies
$$
     S^2 = p_1(\mathcal{F})+rp_2(\mathcal{F})+r^2p_3(\mathcal{F})+r^3p_4(\mathcal{F})=\mathcal{F}.
$$
So there is a third-order polynomial in $\mathcal{F}$ that is a square root of $\mathcal{F}$.
A: I think what you are looking for is the Fractional Fourier Transform, which is a generalization of the Fourier transform to the $n^{\textrm{th}}$ power. They are a subset of the Linear Canonical Transforms.
For any value $\alpha=\frac{\pi}{2} \cdot n$, the definition of the Fractional Fourier Transform of a function $f$ is
$$
\mathcal{F}_\alpha[f](u) = 
\sqrt{1-i\cot(\alpha)} \cdot e^{i \pi \cot(\alpha) u^2} 
\int_{-\infty}^\infty 
e^{-i2\pi \left(\csc(\alpha) u x - \frac{\cot(\alpha)}{2} x^2\right)}
f(x)\, \mathrm{d}x
$$
For the square root, filling in $\alpha=\frac{\pi}{4}$ gives:
$$
\mathcal{F}_\alpha[f](u) = 
\sqrt{1-i} \cdot e^{i \pi  u^2} 
\int_{-\infty}^\infty 
e^{-i2\pi \left(\sqrt{2} u x - \frac{1}{2} x^2\right)}
f(x)\, \mathrm{d}x
$$
For integer multiples of $\pi$, the cotangent and cosecant functions diverge, so you need to take the limit instead.
