Problem with showing $\lim_{n\rightarrow \infty} \int_A \cos(nxy) \, d\lambda_2=0$ I need to show that $$\lim_{n\rightarrow \infty} \int\limits_A \cos(nxy) \, d\lambda_2=0$$ for every Borel set $A\subset \mathbb{R}^2$ which has finite Lebesgue measure.
I tried to use the definition of Lebesgue measure, namely
$$ \int \chi_A \;\operatorname d\mu := \mu(A),   $$
but when I am trying to use this fact it is not occurring very helpful in my opinion. Is it a good idea to use Lebesgue's dominated convergence theorem? Then the integrable function $g$ would be equal to one if I am thinking correctly.
Thanks in advance.
 A: Here we present a solution that adapts the proof of Riemann-Lebesgue lemma.
Step 1. We first consider the case where $R = [a, b]\times[c, d]$ is a rectangle. Then by the Fubini-Tonelli theorem,
\begin{align*}
\int_{R} \cos(nxy)\,\mathrm{d}x\mathrm{d}y
&= \int_{c}^{d} \left( \int_{a}^{b} \cos(nxy) \,\mathrm{d}x \right) \mathrm{d}y \\
&= \int_{c}^{d} (b\operatorname{sinc}(nby) - a\operatorname{sinc}(nay)) \mathrm{d}y,
\end{align*}
where $\operatorname{sinc}(x) = \frac{\sin x}{x}$ if $ x \neq 0$ and $\operatorname{sinc}(0) = 1$. Also note that
$$\lim_{|x| \to \infty} \operatorname{sinc}(kx)
= \mathbf{1}_{\{c=0\}}
= \begin{cases}
1, & \text{if } c = 0, \\
0, & \text{if } c \neq 0.
\end{cases} $$
So, letting $n\to\infty$ and applying the dominated convergence theorem (check that this is indeed applicable!), we have
\begin{align*}
\lim_{n\to\infty} \int_{R} \cos(nxy)\,\mathrm{d}x\mathrm{d}y
&= \int_{c}^{d} \lim_{n\to\infty} (b\operatorname{sinc}(nby) - a\operatorname{sinc}(nay)) \mathrm{d}y \\
&= \int_{c}^{d} (b \mathbf{1}_{\{by=0\}} - a \mathbf{1}_{\{ay=0\}} ) \mathrm{d}y \\
&= 0.
\end{align*}
Step 2. Now let $A$ be an arbitrary Borel set of finite measure. Then for each $\epsilon > 0$, there exists finitely many rectangles $R_1,\dots,R_m$ such that
$$ \int_{\mathbb{R}^2} \left| \mathbf{1}_A - \sum_{k=1}^{m} \mathbf{1}_{R_k} \right| \, \mathrm{d}x\mathrm{d}y < \epsilon. $$
This gives
\begin{align*}
&\left| \int_{A} \cos(nxy)\,\mathrm{d}x\mathrm{d}y \right| \\
&= \left| \int_{\mathbb{R}^2} \cos(nxy) \mathbf{1}_A \,\mathrm{d}x\mathrm{d}y \right| \\
&\leq \left| \int_{\mathbb{R}^2} \cos(nxy) \left(\mathbf{1}_A - \sum_{k=1}^{m} \mathbf{1}_{R_k} \right) \,\mathrm{d}x\mathrm{d}y \right| + \sum_{k=1}^{m} \left| \int_{\mathbb{R}^2} \cos(nxy) \mathbf{1}_{R_k} \,\mathrm{d}x\mathrm{d}y \right| \\
&\leq \epsilon + \sum_{k=1}^{m} \left| \int_{\mathbb{R}^2} \cos(nxy) \mathbf{1}_{R_k} \,\mathrm{d}x\mathrm{d}y \right|.
\end{align*}
So letting $\limsup$ as $n\to\infty$, the previous step tells that
$$ \limsup_{n\to\infty} \left| \int_{A} \cos(nxy)\,\mathrm{d}x\mathrm{d}y \right| \leq \epsilon. $$
But since the left-hand side is independent of the choice of $\epsilon > 0$, we may let $\epsilon \downarrow 0$ to find that the limsup is zero, which in turn proves the desired claim.
