Fourier-like operator on $L^2$ is compact For real numbers $a,b>0$, consider the following continuous operator:
$$ \mathcal{G}:=\mathcal{F}^{-1}\chi_{[-b,b]}\mathcal{F}\chi_{[-a,a]}: L^2(\mathbb{R})\to L^2(\mathbb{R}),$$
where the $\chi$'s represent the multiplication operator that multiplies by the corresponding indicator function, and $\mathcal{F}$ is the Fourier transform. How could one prove that it is compact? Moreover, houw could we prove that the following similar-looking operator
$$ \mathcal{F}^{-1}(1-\chi_{[-b,b]})\mathcal{F}\chi_{[-a,a]}: L^2(\mathbb{R})\to L^2(\mathbb{R})$$ isn't compact? For the latter I guess we must find a sequence of functions with bounded $L^2$-norms such that no subsequence of their images converges in $L^2$, and maybe $e^{\pi inx/a}$ would work (since they form an orthonormal basis for $L^2([-a,a])$), where $n\in\mathbb{Z}$, but I'm having some trouble finishing the computations. For the former, a straightforward computation gives:
$$\mathcal{G}f(x) = \frac{1}{\pi}\int_{-a}^a f(y)\frac{\sin(2\pi b(x-y))}{x-y} dy $$
(unless I'm making a mistake). So it is (up to the $1/\pi$ constant) an integral operator with kernel 
$$ K(x,y) = \frac{\sin(2\pi b(x-y))}{x-y} \chi_{|y|\le a}, $$
and I read somewhere that such an integral operator is compact if an only if its kernel is in $L^2$. In this case, this would reduce the problem to proving that
$$\int_{\mathbb{R}}\int_{-a}^a \bigg|\frac{\sin(2\pi b(x-y))}{x-y}\bigg|^2 dy dx <\infty,$$
but is this theorem I cited true? If so, how could one prove it? I couldn't find a proof online.
Thank you so much for your help!
 A: To see compactness what you can do is start with an arbitrary sequence $x_n$ whose norm is bounded and then see that $Tx_n$ admits a limit point, where $T$ is the operator in question.
Well, if I should do this I don't like the form your operator is in. I will massage it a bit to get something nicer, the compactness of the massaged thing will be equivalent to the compactness of the original operator. The operator I want to look at is:
$$T:L^2(\Bbb R)\to L^2(\Bbb R), \quad x \mapsto\left( t\mapsto \chi_{[-a,a]}(t) \int_{\Bbb R}\frac{\sin(b(y-t))}{y-t} x(y)\,dy\right).$$
Why is this more or less the same operator? Well this operator is nothing other than (up to a constant $2\pi$ or something like that) $x\mapsto \chi_{[-a,a]}\ \cdot (\mathcal F(\chi_{[-b,b]}) * x)$ where $*$ is the convolution. Now we can use a calculational rule of the convolution with the Fourier transform to get that
$$T(x)= \chi_{[-a,a]} \cdot [\mathcal F(\chi_{[-b,b]}) * \mathcal F(\mathcal F^{-1}(x))] = \chi_{[-a,a]}\cdot \mathcal F[ \chi_{[-b,b]}\ \cdot \mathcal F^{-1}(x)]$$
well this is your original map except the order is reversed. Or you could also say that your original map is $\mathcal F^{-1}\circ T \circ \mathcal F$ (with the role of the constants $a,b$ swapped), ie it is unitarily equivalent to $T$. Hence compactness of $T$ is the same as compactness of your original map.
Why is $T$ compact? Let $x_n$ be some norm bounded sequence in $L^2(\Bbb R)$. Since Hilbert spaces are reflexive there is some weak limit point of this sequence, call it $x$ and pass to a subsequence so that $x_n$ converges weakly to $x$. We will show that $Tx_n$ converges in norm to $Tx$. The first step is to show that $\|Tx_n\|^2\to \|Tx\|^2$. Here:
$$\|Tx_n\|^2 = \int_b^b \left|\int_{\Bbb R}\frac{\sin(b(y-t))}{y-t} x_n(y)\ dy\right|^2\ dt$$
Well the inner integral is a scalar product of an $L^2$ function with $x_n$ and hence converges to $\int_{\Bbb R} \frac{\sin(b(y-t)}{y-t} x(y)\ dy$ as $n\to\infty$ for all $t$ (remember that $x_n$ converges weakly to $x$). Furthermore the inner integral is bounded by $\left|\int_{\Bbb R} \frac{\sin(by)}y\ dy\right| \cdot \|x_n\|$ for all $t$, hence bounded by $C\cdot \chi_{[-b,b]}(t)$ for an appropriate constant $C$. By the Lebesgue dominated convergence theorem you then get that the entire expression converges to $\|Tx\|^2$.
Now:
$$\|Tx-Tx_n\|^2 = \|Tx\|^2 + \|Tx_n\|^2 - \langle Tx, Tx_n\rangle - \langle Tx_n, T x\rangle$$
Since $T$ is norm continuous it is weak continuous and $Tx_n$ converges weakly to $Tx$. Hence this entire expression converges to
$$\|Tx\|^2+\|Tx\|^2- \langle Tx,Tx\rangle - \langle Tx, Tx\rangle =0.$$
A: Hints: let $\{f_n\}$ be a uniformly bounded sequence in $L^2(\mathbb{R}^n)$. Let $g_n = \mathcal{F}\chi_{[-a,a]}f_n$. Note that each $g_n$ is smooth.


*

*Show that the $g_n$ are uniformly bounded in $L^{\infty}$. (Hint: get an $L^1$ estimate on $\chi_{[-a,a]}f_n$.)

*Show that the derivatives $g_n'$ are uniformly bounded. (This should not be that different from step 1.) Conclude that the sequence $\{g_n\}$ is uniformly equicontinuous.

*The Arzela-Ascoli theorem says there is a subsequence $\{g_{n_k}\}$ such that $\chi_{[-b,b]}g_{n_k}$ converges uniformly in $L^{\infty}$. Use this to conclude that $\mathcal{F}^{-1}\chi_{[-b,b]}\mathcal{F}\chi_{[-a,a]}f_{n_k}$ converges in $L^2$.

A: In general the following is true: if $\Omega$ is an open subset of $\mathbb{R}^n$ (for the sake of simplicity in this proof we just take $\Omega=\mathbb{R}^n$, which is the case of the problem's statement), and $K\in L^2(\Omega\times\Omega)$, then $T: L^2(\Omega)\to L^2(\Omega)$ defined via $(Tf)(x) = \int_{\Omega} K(x,y)f(y) dy$, is a compact continuous linear operator. First, notice it's well-defined (i.e. $(Tf)(x)$ converges for ae $x$) because $|(Tf)(x)| \le ||K(x,\bullet)f(\bullet)||_{L^1}\le ||K(x,\bullet)||_{L^2}||f||_{L^2}$ and $||K(x,\bullet)||_{L^2}<\infty$ for ae $x$, otherwise $\int |K(x,y)|^2 dy =\infty$ for all $x$ in some set $S$ of positive measure, so $\int\int |K(x,y)|^2 dy dx \ge \int_{x\in S}\int |K(x,y)|^2 dy dx=\infty$, contradicting $K\in L^2$. Clearly $T$ is linear, and 
$$||Tf||^2_{L^2} \le \int \left( \int |K(x,y)f(y)| dy\right)^2 dx \le \int \left(\int |K(x,y)|^2 dy\right)\left(\int |f(y)|^2 dy\right) dx \le ||K||^2_{L^2}||f||^2_{L^2},$$
i.e. $||T||\le ||K||_{L^2}$ so $T$ is continuous. Finally we prove the hard part: $T$ it compact. Let $\{\phi_n(x)\}_{n=1}^\infty$ be an orthonormal basis for $L^2(\Omega)$, then $\{\phi_n(x)\overline{\phi_m(x)}\}_{n,m=1}^\infty$ is an orthonormal basis for $L^2(\Omega\times\Omega)$, so we may write $K(x,y) = \lim_{N\to\infty}\sum_{n,m\le N}\alpha_{n,m}\phi_n(x)\overline{\phi_m(y)}$, with the limit in $L^2$. Let $T_N$ denote the operator with kernel $\sum_{n,m\le N}\alpha_{n,m}\phi_n(x)\overline{\phi_m(y)}$, we prove that $T_N\to T$ in norm. Since each $T_N$ is of finite rank (since its image is contained in the span of $\phi_1,\ldots,\phi_N$), this will prove $T$ is compact, as desired. We have 
$$ ||T_N-T||\le ||K(x,y) - \sum_{n,m\le N}\alpha_{n,m}\phi_n(x)\overline{\phi_m(y)}||_{L^2}$$
by the same computation as above. The RHS goes to 0 as $N\to\infty$ so the LHS also goes to $0$, QED.

This reduces the problem to proving the integral at the end of the problem indeed converges, which is equivalent to proving
$$\int_{(x,y)\in[-R,R]\times [-a,a]} \bigg|\frac{\sin(2\pi b(x-y))}{x-y}\bigg|^2 dy dx < C,$$
for some $C$ independent of $R$ and $R\to\infty$. Extend the region of integration to the parallelogram with vertices $(-R-2a, -a), (-R, a), (R+2a, a), (R, -a)$. With $z=x-y$, the integral is bounded by 
$$ 2\sqrt{2}a\int_{-R-a}^{R+a}\bigg|\frac{\sin(2\pi bz)}{z}\bigg|^2 dz.$$
After re-scaling $z$ and letting $R\to\infty$, we just have to prove that 
$$\int_{-\infty}^\infty \bigg|\frac{\sin z}{z}\bigg|^2 dz <\infty.$$
Away from $z=0$ this converges since it's dominated by $1/z^2$, and near $z=0$ we use the taylor series to see that we're integrating $O(z^4)$ so it obviously converges. Alternatively near $z=0$ the function is continuous so it must be locally integrable at 0, QED.

Finally, the other operator isn't compact because if it were, then the addition of two compact operators is compact so $\chi_{[-a,a]}$ would be compact. But this acts like the identity on $L^2([-a,a])$ which is an infinite-dimensional Hilbert space, and thus the identity operator on this space cannot be compact, QED.
