Calculate the definite integral with $\lim$ Let some function $f \in C[a, b]$.
Find $\lim \limits_{n \to \infty} \int \limits_a^b  \cos(nx) \cdot 
 f(x) dx$.
What is the generic way solve $\lim \limits_{n \to \infty} \int \limits_a^b g(nx)\cdot f(x) dx$, where $g(nx)$ is some trigonometric function?
It seems it can be tackled using the concept of improper integral.
 A: I don't know if this helps but $$\lim_{n \to \infty} \int_{a}^{b} f(x) g(x) dx =$$ $$ \lim_{n \to \infty } \left[ f(x) \int_a^b g(x) dx-\int_a^b \left(\int g(x) dx \right) f'(x) dx\right]$$
If $g(x)$ reduces to a constant after $n$ derivatives then repeated application of integration by parts reducing it to integration of $f(x)$. Making it just a repeated integration of the trig function. If the $f(x) \in C^{\infty}$ like a polynomial for examle.
A: As Paramanand Singh noted, 
$$ \int_a^b \cos(nx) f(x) dx $$ 
is a kind of Fourier coefficient / transform, and vanishes for $n\to \infty$. The proof is pretty short with dominated convergence theorem. 
We extend $f$ onto $\mathbb R$ by setting it to $0$ outside $[a,b]$ for notational convenience. Then, substituting $x$ with $x + \frac{\pi}n$ yields
\begin{align*}
 2\biggl| \int \cos(nx) f(x) dx \biggr| 
&= \biggl| \int \cos(nx) f(x) dx + \int \underbrace{\cos(nx + \pi)}_{=-\cos(nx)} f(x + \tfrac{\pi}n) dx \biggr| \\
&\le \int |\cos(nx)| |f(x) - f(x + \tfrac{\pi}n)| dx \\
&\le \int |f(x) - f(x + \tfrac{\pi}n)| dx \to 0,
\end{align*}
as $f$ is integrable and continuous on $\mathbb R \setminus \{a,b\}$.
Notes: Instead of using dominated convergence, you can also extend $f$ on the left with $f(a)$ and on the right with $f(b)$ and exploit that $f$ is uniformly continuous and bounded.
