Limit of a composite function approaching infinity. I always have problem in applying limits in a composite function. The theorem says that
if $f$ is continuous at $b$ and $\lim_{x \to a}g(x) = b$, then $\lim_{x \to a}f(g(x)) = f(\lim_{x \to a}g(x))$. But I never understood this theorem completely. For example, I tried to solve the following question using the theorem but the answer I got was wrong.
$$
\lim_{x \to \infty} \cos^2\Bigl(\pi\sqrt[3]{n^3+n^2+2n}\Bigr) = \cos^2\Bigl(\lim_{x \to \infty}\Bigl(\pi\sqrt[3]{n^3+n^2+2n}\Bigr)\Bigr)
$$
 A: The theorem you're stating is the definition of continuity for a function. Notice that if
$$\lim_{x \to a}f(g(x)) = f(\underbrace{\lim_{x \to a}g(x)}_{\color{blue}{b}}) = f(b)
$$
you're just saying that $f(g(x))$ is approaching $f(b)$ as it gets closer to said value.

Now, notice that if you want to apply this to your question, you'd be implicitly assuming that $\sqrt[3]{n^3+n^2+2n}$ is a continuous function. With this assumption, it is true that
$$
\lim_{n \to \infty } \sqrt[3]{n^3+n^2+2n} = \infty
$$
but since $\cos^2(\infty)$ is undefined, if you're dealing with a continuous limit then the limit does not exist.

There is one more possible interpretation of your problem. In this assumption, you're assuming that $\sqrt[3]{n^3+n^2+2n}$ is a sequence instead of a function. This means you're only evaluating $\sqrt[3]{n^3+n^2+2n}$ at values of $n \in \mathbb{N}$, instead of any real value. In this case, you'd be working with a limit of a sequence instead of the limit of a function, where here you can get a finite answer as follows.
You can show that
$$
\sqrt[3]{n^3+n^2+2n} \sim n + \frac{1}{3} + \mathcal{O}\left(\frac{1}{n}\right)
$$
as $n \to \infty$. You can show this taking the Laurent expansion of $\sqrt[3]{n^3+n^2+2n}$ at infinity, for example. From here we would see that
\begin{align}
\lim_{n \to \infty} \cos^2\left(\pi\sqrt[3]{n^3+n^2+2n}\right)  &=  \lim_{n \to \infty} \cos^2\left(\pi n + \frac{\pi}{3} \right)=  \left[\frac{1}{2}\lim_{n \to \infty}\cos\left(\pi n\right) - \frac{\sqrt{3}}{2}\lim_{n \to \infty} \sin\left(\pi n\right)\right]^2
\end{align}
And here's where you use the hypothesis that you're just taking integer values. Since $n \in \mathbb{N}$, then $\sin(\pi n) = 0$ and $\cos(\pi n) = (-1)^n$ for any $n \in \mathbb{N}$, even if we take the limit. With this we then get that
$$
\lim_{n \to \infty} \cos^2\left(\pi\sqrt[3]{n^3+n^2+2n}\right)   =  \left[\frac{1}{2}(-1)^n- \frac{\sqrt{3}}{2} (0)\right]^2= \boxed{\frac{1}{4}}
$$
where you get your desired result.
Note that the limits $\lim_{n \to \infty}\sin\left(\pi n\right)$ and $\lim_{n \to \infty}\cos\left(\pi n\right)$ don't exist if $n$ can be any real value as both these functions oscillate infinitely between $-1$ and $1$, but by restricting the values of $n$ to only integer values we can give some values to these limits.
A: Suppose $f$ is continuous at $c$ and defined in a suitable neighborhood of $c$. Then, if $\lim_{x\to a}g(x)=c$, we can deduce that
$$
\lim_{x\to a}f(g(x))=f(c)
$$
so you can “push in” the limit. This can also be extended to the case when $c$ is infinity, but in this case you need that $\lim_{y\to\infty}f(y)$ exists.
The case of sequences is similar. But in your case $\lim_{y\to\infty}\cos^2y$ does not exist.
How do you deal with this limit? The fact that you have the cosine squared is the key. Indeed, you can see that
$$
\cos^2(\pi x_n)=\cos^2(\pi x_n-\pi n)
$$
In your case $x_n=\sqrt[3]{n^3+n^2+2n}$ and
$$
x_n-n=\frac{n^3+n^2+2n-n^3}{\sqrt[3]{(n^3+n^2+2n)^2}+n\sqrt[3]{n^3+n^2+2n}+n^2}
$$
and you can use the trick of pulling $n^3$ from the cube roots in the denominator to get
$$
x_n-n=\frac{1+(2/n)}{\sqrt[3]{(1+(1/n)+(2/n^2))^2}+\sqrt[3]{1+(1/n)+(2/n^2)}+1}
$$
Now
$$
\lim_{n\to\infty}\pi(x_n-n)=\pi/3
$$
and you can push the limit inside:
$$
\lim_{n\to\infty}\cos^2(x_n)=\lim_{n\to\infty}\cos^2(\pi(x_n-n))=
\cos^2\Bigl(\lim_{n\to\infty}\pi(x_n-n)\Bigr)=\cos^2\frac{\pi}{3}=\frac{1}{4}
$$
