Is there a differential limit?

I'm wondering if there's such a concept as a "differential limit". Let me give an example because my nomenclature is my own and unofficial, but hopefully indicative of the concept.

For some function f(x), there may exist a derivative of that function f'(x) which we call a first order differentiation of f(x). Likewise from f'(x) we could also derive a second order function f''(x). Is there a mathematical concept of the limit of a function as the order of differentiation goes to infinity?

I can think of a function that I can take the derivative of an infinite number of times ( albeit without getting a constant value): f(x) = sin(x)

The function g(x) = x^2 might have a differential limit of 0. (g'(x) = 2x; g''(x) = 2; g'''(x) = 0 ...)

Is there any usefulness to this concept? A topic of study or keyword that discusses this? Any more examples of infinitely derivable functions?

Most importantly if this is interesting to anyone other than myself: what is a good resource for learning more?

-
We use infinitely differentiable functions often in mathematics. We call it the set $C^{\infty}$. A function that is infinitely differentiable is called "smooth" mathworld.wolfram.com/C-InfinityFunction.html – tomcuchta Jun 13 '12 at 3:35

First: yes, we care about how many times a function can be differentiated. In fact, we have names for such classes: $\mathcal{C}_0$ is the collection of all continuous functions, $\mathcal{C}_1$ the class of functions that have continuous derivative; $\mathcal{C}_2$ the class of functions that have continuous second derivative; and so on. We have that $$\mathcal{C}_0 \supsetneq \mathcal{C}_1 \supsetneq\mathcal{C}_2\supsetneq\mathcal{C}_3\supsetneq \cdots \supsetneq \mathcal{C}_n\supsetneq \cdots$$

Then we have "infinitely differentiable functions", functions that have continuous derivatives of all orders; this is denoted $\mathcal{C}_{\infty}$. There are functions that have derivatives of all orders that don't cycle through values, such as $e^{x^2}$.

Even beyond the infinitely differentiable functions we have the notion of "analytic function", which are functions that can be defined at every point by a power series that converges on an open interval containing the point.

Now, I think you are talking about trying to make sense of a "limit" of the functions as you take derivatives. So that for a polynomial the "limit" should be $0$, a function like $\sin x$ should not have a limit, and so on.

There are in fact several different ways of making the idea of "differential limit" precise. I'm going to talk more generally about what "limit" and "convergence" can mean here, before addressing the specific construction you are talking about. I'll spill the beans and say that I haven't really seen this particular construction before (there's too much to wade through below, and I don't want you to be annoyed when you reach the end and find that I say "I haven't seen it...") But that does not mean you can't ask interesting questions about it.

What you describe is a sequence of functions. The main issue is that one has to define what one means by saying that two functions are "close to one another". Then you get the notion of convergence of functions.

Turns out that there are many different ways of saying this, and which one is useful depends on the context.

Let's take the case of a sequence of real valued functions of real variable, with domain all of $\mathbb{R}$. Let's call the sequence $f_n$ (in your case, $f_n = \frac{d^nf}{dx^n}$ for a given $f$).

1. The most natural way of saying that the sequence $f_n$ converges to a function $f$ is:

$f_n\to f$ if for every $x$, $\lim\limits_{n\to\infty}f_n(x) = f(x)$.

This is called pointwise convergence. For example, the sequence $f_n(x) = x^n$ defined on $[0,1]$ converges pointwise to the function $$f(x) = \left\{\begin{array}{ll} 0 &\text{if }0\leq x\lt 1\\ 1 & \text{if }x=1. \end{array}\right.$$

It turns out that pointwise convergence is natural, but is not very "good", in the sense that a lot of properties we find interesting about functions are not preserved by pointwise convergence. For example, above, all functions $f_n$ are continuous, but their limit is not.

The problem arises because we are forcing no connection between how $f_n(x)$ converges to $f(x)$ and how $f_n(y)$ converges to $f(y)$ with $x\neq y$; that means that convergence on some points can "lag behind" convergence in points very close to them. This can be remedied by making things a bit tighter. If we write out the definition of the limits above, we obtain the following description:

$\{f_n\}$ converge pointwise to $f$ if and only if for every $x$, for every $\epsilon\gt 0$, there exists $N$, which may depend on both $x$ and $\epsilon$, such that for all $n\geq N$ we have $|f_n(x)-f(x)|\lt \epsilon$.

The way to make this tighter, to make sure the the $f_n(x)$ converge to $f$ more or less "the same way", is to force the $N$ to depend only on $\epsilon$, and not on $x$ as well. This is accomplished with:

2. We say that $\{f_n\}$ converge uniformly to $f$ if and only if for every $\epsilon\gt 0$ there exists $N$, which depends only on $\epsilon$, such that for all $x$ and all $n\geq N$ we have $|f_n(x)-f(x)|\lt\epsilon$.

Uniform convergence is better than pointwise convergence: if every element of the sequence is continuous, and the sequence converges uniformly, then the limit is continuous, for example.

There are yet other ways of talking about functions converging. Here are a few:

1. Fix a real number $p$, $1\leq p$. We say that $\{f_n\}$ converges to $f$ in $p$-norm if and only if $$\lim_{n\to\infty}\int_{\infty}^{\infty}|f_n(x)-f(x)|^p\,dx = 0.$$

2. Closely related: we say that $\{f_n\}$ converges to $f$ in the sup-norm if and only if $$\lim_{n\to\infty}\sup\{|f_n(x)-f(x)|\} = 0.$$

3. There is a way of measuring sets of real numbers, called the Lebesgue measure, $\lambda$, the details of which are perhaps too much to go over now. But intuitively, it is a way of assigning a "size" to a lot of sets of real numbers, including all intervals and much more complicated sets, in a nice and consistent way. We say that $\{f_n\}$ converges in measure to $f$ if and only if for every $\epsilon\gt 0$, $$\lim_{n\to\infty}\lambda(\{x\mid |f_n(x)-f(x)|\geq\epsilon\}) = 0.$$

4. We say that $\{f_n\}$ converges almost uniformly to $f$ if and only if for every $\epsilon\gt 0$ there is a set $X\subseteq \mathbb{R}$, with $\lambda(X)\lt\epsilon$, such that $\{f_n\}$ converges uniformly to $f$ on $\mathbb{R}-X$.

5. There are small variations to each of those; depending on context, one may be able to define other ways of convergence, or some of the above may no longer make sense. A typical variation is to take any of the above types of convergence (except almost uniform, for which the concept doesn't add anything), and add the condition "almost everywhere" (or "almost surely" if you are doing probability). That means that there exists a set $Y$ contained in the domain, of measure $0$, such that the type of convergence you are interested in occurs in $\mathbb{R}-Y$. (Sets of measure $0$ need not be empty, and they can be quite complicated: for example, any countable set has measure $0$, and there are uncountable sets that do as well, like the Cantor ternary set.

Another variation is to define the concept of "Cauchy sequence" for each of the types, which means making the analog of the definition of "Cauchy sequence" of real numbers: instead of comparing to a limit, we ask that for all $n$ and $m$ greater than $N$, $f_n$ and $f_m$ have the relevant property. For instance, "pointwise Cauchy" means that for all $x$ and all $\epsilon\gt 0$ there exists $N$ such that for all $n,m\gt N$ we have $|f_n(x)-f_m(x)|\lt \delta$. "Cauchy in measure" means that for all $\epsilon\gt 0$ and all $\delta\gt 0$ there exists $N$ such that if $n,m\gt N$, then $\lambda(\{x\mid |f_n(x)-f_m(x)|\geq\epsilon\}\lt\delta$; and so on.

All of these concepts are extremely important and useful. They show up in probability, statistics, functional analysis, analysis, and other areas. There are known implications (e.g., uniform convergence implies pointwise convergence, and so on).

Now, what about the specific case in which we start with a function $f_0$ (necessarily in $\mathcal{C}_{\infty}$) and we define $f_n$ to be the $n$th derivative of $f_0$? I confess I haven't seen this particular sequence addressed, but you can ask all the questions related to the convergence types above: is the sequence pointwise Cauchy? $p$-norm Cauchy? Cauchy in measure? Does it converge uniformly? Almost uniformly? Pointwise? In measure? And so on. Seems like an interesting question, if nothing else.

-
Thank you for the thorough answer, that was exactly what I was looking for. – jpredham Jun 13 '12 at 8:58
I think in your first #2 there is a slight typo - we should have that for $n\ge N$ the property holds. – process91 Jun 13 '12 at 13:54
@Michael: Yes, indeed. Thank you. – Arturo Magidin Jun 13 '12 at 15:25

It would seem that if the sequence $f$, $f'$, $f''$, ..., $f^{(n)}$, ... tends to a limit as a function, then that limit would be a solution of the differential equation $\frac{dy}{dx}=y$. So if the limit exists, it is either zero or some multiple of $e^x$. (I'm assuming that the notion of limit we're using is one that the differentiation operator is continuous with respect to, which doesn't seem like much to ask. Pointwise convergence is out, though).

Also, if we have some function whose iterated derivatives converge towards $ce^x$, we can subtract $ce^x$ from the initial function and get a new sequence whose iterated derivatives converge towards the zero function. So if we look for solutions to $\lim_{n\to\infty} f^{(n)}=0$ we essentially get all functions whose iterated derivatives have a limit.

All polynomials qualify, of course, but there are also non-polynomial analytic solutions such as $\sum_{n=0}^\infty \frac{1}{n\cdot n!} x^n$ and in general $\sum_{n=0}^\infty \frac{a_n}{n!}x^n$ whenever $\lim_{n\to \infty} a_n = 0$. I wonder whether some of these have nice closed forms.

-