Bringing limits inside functions when limits go to infinity A standard result says that under suitable conditions to make sure the functions are defined where they need to be, we can write
$$\lim_{x \to c} f(g(x)) = f \left( \lim_{x \to c} g(x) \right)$$
as long as $f$ is continuous at $\lim_{x \to c} g(x)$.
But if the inside function is going to infinity, the continuity condition makes no sense, yet it is still common to say something like
$$\lim_{x \to c} f(g(x)) = \lim_{t \to \infty} f(t) $$
This is more the "substitution" method of evaluating the limit, which I also see used in the standard case instead of formally bringing limits inside of functions. But I've never liked it, because it never felt like substituting was using a limit theorem which I knew to be true in all situations.
My question: what is the most general version of the theorem here? In the second formula, does something need to be continuous in some sense?
 A: Let me bring one theorem from Kudriavtsev L.D. "Course of Mathematical Analysis",  4.8*, 1 tom. 1981, page 108.
Assume exists finite or infinite limits $\lim\limits_{x\to a}f(x)=b$ and $\lim\limits_{y\to b}F(y)$. Also let's assume, that exists some deleted neighborhood of $a$ in which holds $f(x)\ne b$. Then exists limit of compound function $F\circ f$ and holds
$$\lim\limits_{x\to a}F(f(x))=\lim\limits_{y\to b}F(y)$$
Counterexample if we omit $f(x)\ne b$ condition:
let's assume $\forall x \in \mathbb{R}, f(x)\equiv0 $ and
$$F(y)=\begin{cases} 0 ,& y\ne 0\\
1 , & y=0
\end{cases}$$
then $\lim\limits_{y\to 0}F(y)=0$ and $\lim\limits_{x\to 0}F(f(x))=1$. Such example is impossible if, as written above, we add condition $x\ne 0\Rightarrow f(x)\ne 0$.
Note. If $f$ is one-to-one map from some deleted neighborhood $\mathring{U}(b,\varepsilon)$ to some deleted neighborhood $\mathring{U}(a,\delta)$ and we have $\lim\limits_{x\to a}f(x)=b$ and  $\lim\limits_{y\to b}f^{-1}(y)=a$,  then $\lim\limits_{y\to b}F(y)$ exists if and only  if exists $\lim\limits_{x\to a}F(f(x))$. If they exist, then they are equal.
A: For reference, Theorem 1: If $\lim_{x \to c} g(x)$ exists, and $f$ is continuous at that value $\lim_{x \to c} g(x)$, then
$$ \lim_{x \to c} f(g(x)) = f \!\left(\lim_{x \to c} g(x)\right)$$
You're right to question this sort of thing! The version involving infinity is not a trivially direct result of Theorem 1. The sufficient conditions for the equality become unclear, since as you noted "continuous at infinity" is nonsense.
But it is true that if $\lim_{x \to c} g(x) = +\infty$ and $\lim_{y \to +\infty} f(y)$ exists, then $$\lim_{x \to c} f(g(x)) = \lim_{y \to +\infty} f(y)$$
Of course, in this statement even the meanings of the two limit preconditions are different from the ordinary meaning of a function's limit at a point. "$\lim \ldots = +\infty$" uses "$\forall N \ldots \Rightarrow f(x)>N$" instead of "$\forall \epsilon>0 \ldots \Rightarrow |f(x)-L| < \epsilon$"; and "$\lim_{x\to +\infty}$" uses "$\exists M: x>M \Rightarrow"$ instead of "$\exists \delta: |x-c|<\delta \Rightarrow$".
This version is easily proved from those $\delta$-$N$ and $M$-$\epsilon$ limit definitions, and that proof looks very much like the proof of Theorem 1. One difference which does come up is that in the original proof, with $\lim_{x \to c} g(x) = L$, it's possible to have $0<|x-c|<\delta$ and $|g(x)-L|=0$, which is why continuity of $f$ is required: If we only know that the limit of $f$ at $L$ exists, this tells us something about $f(y)$ when $0<|y-L|<r$ but we also need to cover the case $y=L$. But when $L$ is replaced with $+\infty$, there's no case $g(x)=+\infty$ or $y=+\infty$ to worry about.

One reason many of these limit-related statements do turn out so similar can be somewhat explained by this construction:
Let $X$ be the topology whose elements are the real numbers, the symbol $+\infty$, and the symbol $-\infty$, and whose open sets are sets $U \subseteq X$ satisfying:

*

*For every real number $x \in U$, there exists a real $\delta>0$ such that for every real $y$, $|y-x|<\delta \Rightarrow y \in U$. (In other words, $(x-\delta, x+\delta) \subseteq U$.)

*If $+\infty \in U$, then there exists a real $M$ such that for every real $y$, $y>M \Rightarrow y \in U$. (In other words, $(M,+\infty) \subseteq U$.)

*If $-\infty \in U$, then there exists a real $M$ such that for every real $y$, $y<M \Rightarrow y \in U$. (In other words, $(-\infty,M) \subseteq U$.)

It's straightforward to show this $X$ is actually a topology. Notice that any open set in $\mathbb{R}$ is also an open set in $X$. (It cannot contain $+\infty$ or $-\infty$, and rule 1 above defines the open sets of $\mathbb{R}$.)
Then we can define limits which do involve $+\infty$ or $-\infty$ by saying that if $a \in X$ and $L \in X$, then the equation $\lim_{x \to a} f(x) = L$ means: For every open set $V$ containing $L$, there is an open set $U$ containing $a$ such that $f(U\setminus\{a\}) \subseteq V$. As for the classic limit, we then say "the limit exists" if some element $L \in X$ makes that equation true. We can prove that the equation cannot be true for two different values of $L$. When the limit does exist, we associate the expression $\lim_{x \to a} f(x)$ in other general contexts with that unique value $L$.
And now within this topology $X$, we actually can say a function "has an infinite value at $c$", or "has a value at $-\infty$", or "is continuous at $+\infty$". Care is still needed though! We can't add, subtract, multiply or divide variables in $X$, since there's no making sense of $+\infty-(+\infty)$ or $\frac{+\infty}{+\infty}$. (Of course, we might be able to prove the variables are actually real numbers, and then do those operations in $\mathbb{R}$.) We do at least have consistent inequalities.
A continuous function $f$ defined on $\mathbb{R}$ or a subset of $\mathbb{R}$ can often be extended to a continuous function $\bar{f}$ on a larger subset of $X$ than the original domain, possibly all of $X$. This makes using those functions valid with a variable approaching an infinite value and/or a limit equal to an infinite value. For example, $f(x) = e^x$ extends to be continuous on all $X$ by choosing $\bar{f}(-\infty)=0$ and $\bar{f}(+\infty)=+\infty$. With $f(x) = 1/x^2$ where $x \neq 0$, $\bar{f}$ can be continuous on all $X$ by defining $\bar{f}(-\infty)=\bar{f}(+\infty)=0$ and $\bar{f}(0) = +\infty$. The similar function $f(x) = 1/x$ where $x \neq 0$ can't be extended at $0$. But the function $f(x) = 1/x$ on domain $x>0$ can be extended to $[0,+\infty]$ by choosing $\bar{f}(0)=+\infty$ and $\bar{f}(+\infty)=0$. The function $f(x) = \sin x$ can't be extended in this way.
Using this $X$ would make it possible to state and write the most common and simple limit theorems just once, instead of the many variations implied but often not pointed out when infinities are involved. (One-sided limits at interior real points of a domain are a different issue, which may still require separate proofs or further generalization.)
A: I have a distant memory, over 15 years ago, of reading an old textbook with a more general framework, in which this theorem can be proven in great generality. The following is based on the fragments of what I could recall. I make no guarantee that my nomenclature is standard (nor do guarantee that it's unique!).
Suppose $D \subseteq T \setminus \{\emptyset\}$ has the following properties:

*

*For any $U_1, U_2 \in D$, there exists some $U_3 \in D$ such that $U_3 \subseteq U_1 \cap U_2$ ($D$ is a downwards-directed set with respect to set inclusion), and

*For any $U \in D$, there exists some $U' \in D$ such that $\operatorname{cl} U' \subseteq U$.

We will call such a set $D$ a direction. We can use this concept to, very generally, state and prove the property you're talking about.
Some examples of directions in $\Bbb{R}$:

*

*For any $a \in \Bbb{R}$, the set $\{(a - \delta) \cup (a + \delta) : \delta > 0\}$ is a direction, which we denote simply by $a$.

*The sets $\{(a, a + \delta) : \delta > 0\}$ and $\{(a - \delta, a) : \delta > 0\}$ are both directions, denoted by $a^+$ and $a^-$ respectively.

*We also denote the directions $\{(x, \infty) : x \in \Bbb{R}\}$ and $\{(-\infty, x) : x \in \Bbb{R}\}$ by $\infty$ and $-\infty$ respectively.

*The set $\{(-\infty, x) \cup (x, \infty) : x > 0\}$ is also a direction, which we could notate $\pm \infty$.

These encapsulate the idea of $x \to a$, $x \to a^+$, $x \to a^-$, $x \to \infty$, $x \to -\infty$, and $|x| \to \infty$ respectively.
We can define limits with directions too. Specifically, rather than saying $x$ approaches a number, we will say $x$ approaches a direction. The limit too will be a direction. We aim to have $\lim_{x \to a} f(x) = L$ mean the same thing as it normally would, even if $a$ is replaced by $a^+, a^-, \infty, -\infty$, etc. But, this is more flexible: with one definition, we define one-sided limits and infinite limits, both in terms of the dependent variable $x$ and the independent variable $f(x)$.
Suppose we have topological spaces $(X_1, T_1)$ and $(X_2, T_2)$ (again, feel free to sub in $\Bbb{R}$ to both), and $D_i$ is a direction in $(X_i, T_i)$ for $i = 1, 2$. Further, suppose $f : X_1 \to X_2$. Then we say
$$\lim_{x \to D_1} f(x) = D_2$$
if, for all $\mathcal{V} \in D_2$, there exists some $\mathcal{U} \in D_1$ such that
$$x \in \mathcal{U} \implies f(x) \in \operatorname{cl} \mathcal{V}.$$
I hope it's clear that, in the case of $\Bbb{R}$, making $D_1 = a$ and $D_2 = L$, we get the usual definition of $\lim_{x \to a} f(x) = L$. With some work, you could verify that all the various definitions possible are all faithfully encapsulated with this framework.
With that in mind, we can prove the following general theorem:
Theorem. Suppose $(X_i, T_i)$ is a topological space, and $D_i$ is a direction in that space, for $i = 1, 2, 3$. Further, suppose $g : X_1 \to X_2$ and $f : X_2 \to X_3$. Then,
$$\lim_{x \to D_1} g(x) = D_2 \text{ and } \lim_{x \to D_2} f(x) = D_3 \implies \lim_{x \to D_1} f(g(x)) = D_3.$$
Proof. Suppose $\mathcal{W} \in D_3$. Since $\lim_{x \to D_2} f(x) = D_3$, there exists some $\mathcal{V} \in D_2$ such that
$$y \in \mathcal{V} \implies f(y) \in \operatorname{cl} \mathcal{W}.$$
By the second property of directions, there must exist some $\mathcal{V}' \in D_2$ whose closure is contained in $\mathcal{V}$. Using the fact that $\lim_{x \to D_1} g(x) = D_2$, there exists some $\mathcal{U} \in D_1$ such that
$$x \in \mathcal{U} \implies g(x) \in \operatorname{cl}\mathcal{V}' \subseteq \mathcal{V} \implies f(g(x)) \in \operatorname{cl}\mathcal{W}.$$
Thus, by definition, $\lim_{x \to D_1} f(g(x)) = D_3$. $\square$
And that's that! Now, a couple of closing remarks:

*

*Directions can be compared with the relation
$$D_1 \le D_2 \iff \forall U_1 \in D_1, \exists U_2 \in D_2 : U_1 \subseteq U_2.$$
In this way, we see that $a^+, a^- \le a$, or $\infty, -\infty \le \pm \infty$. Moreover, we get nice results like, if $\lim_{x \to D_1} f(x) = D_2$ and $D' \le D_1$, then $\lim_{x \to D'} f(x) = D_2$. Or, if $D' \ge D_2$, then $\lim_{x \to D_1} f(x) = D'$.


*This reveals something important about limits with directions: they are (mostly) not unique! Indeed, using the $\lim_{x \to D_1} f(x) = D_2$ notation is already problematic, as the left hand side does not refer to any one direction. It would probably be better to say $f(x) \to D_2$ as $x \to D_1$ instead.


*The $\le$ relation is a pre-order, i.e. is reflexive and transitive, but not necessarily anti-symmetric. The symmetrised relation,
$$D_1 \sim D_2 \iff D_1 \le D_2 \text{ and } D_2 \le D_1$$
is an equivalence relation, and indeed, it would be fair to consider such directions to be "equivalent". It would mean that one could swap out one direction for an equivalent one in a limit at will. We could, for example, equivalently define the direction $a$ (where $a \in \Bbb{R}$) by $\{(a - 1/n, a + 1/n) : n \in \Bbb{N}\}$ instead, and we would obtain an equivalent direction.


*Be aware that the above definition of a limit assumes a "full" domain, i.e. $g$ is defined everywhere on $X_1$ and $f$ is defined everywhere on $X_2$. This may seem restrictive at first, but remember that we could define our topological space to be a subset of $\Bbb{R}$, and equip it with the subspace topology. If we have $Y \subseteq X$, and $D$ is a direction in $X$, then $D_Y = \{U \cap Y : U \in D\}$ is a direction provided $\emptyset \notin D_Y$. This takes care of real functions whose domain is not the entirety of $\Bbb{R}$.
