Closing up the elementary functions under integration The elementary real-valued functions are not closed under integration.  (Elementary function has a precise definition --  see Risch algorithm in Wikipedia).  This means there are elementary functions whose integrals are not elementary.  So we can construct a larger class of functions by adjoining all the integrals of elementary functions.  You can repeat this process indefinitely.  If I understand things correctly, the set of functions that is the countable closure of this process is closed under integration.  Does any finite iteration of the process achieve closure under integration?
My guess is no.  Has anyone thought about this?
 A: I'm not entirely sure it can be closed; apparently more and more complicated integrals of compositions of elementary functions require further generalizations of the "hypergeometric function" to several variables; for instance, the integral of $\sin(\sin(x))$ requires a Kampé de Fériet hypergeometric function for its closed form, and the elliptic integrals (integrals containing square roots of cubic or quartic polynomials) require the Lauricella or Appell hypergeometrics. As to whether there is "the one hypergeometric to rule them all", I think that's an open question.
EDIT
To clarify the little discussion Qiaochu and I had in the comments for other readers: the reason you should be interested in the hypergeometrics is that virtually all the elementary functions, and combinations thereof (sum, products, compositions, etc.) are expressible as hypergeometrics. As I have shown in the example I gave in the comments, integrating a hypergeometric function with p+q+1 arguments can require a hypergeometric function with p+q+3 arguments. If you imagine this procedure being applied by multiplying a hypergeometric function with each term of a power series, you get even more complicated hypergeometric functions (in that link, you have the multivariate generalization of the hypergeometric function known as Meijer's G function. Integrating a Meijer function can give you a Meijer function with even more arguments).
So, the question one should be asking, I think, is that if there exists "the one hypergeometric to rule them all"; or to (mis)use another colloquialism: "the mother of all hypergeometrics".
EDIT×2
I guess all I have been trying to say, after all this elaboration: unless one would be satisfied with an "it's turtles all the way down" sort of answer, then I think there's no way to satisfactorily resolve this question.
A: In a differential Galois field, integration is the antiderivative operation and not the infinite summation, but I will still call it integration.  
If we start out with simple elementary functions, say rational functions with integer coefficients, then there will be many functions that do not have integrals. If the original set of functions is finite, then only a finite number of functions need to be added.  The process can then be repeated at the next level.  The only catch is that at every stage one has to deal with constants (think of the roots of the denominators). Most functions force the addition of several integrals.  The number of functions added grows geometrically with the levels, so I would expect there is no easy way to define a countable closure.
Are there initial sets for which the process stops? Yes, polynomials for example.  Are there initial sets for which it goes on forever?  Yes. 
A: Alternate between applying the integral to each element and closing the set with respect to the elementary operations.  Let's call the smallest number of such compound iterations necessary to construct a function its "rank".  So exp has rank 0, but erf, erf(exp), and erf + exp all have rank 1.
The integral of a function with rank R may have rank R or R+1.  The derivative of a function with rank R may have rank R or R-1, and the rank of the derivative of a function with rank 0 is always 0.  If the derivative of a function with rank R has rank R-1, let's call that function "primitive" to distinguish between the two phases of the iteration.  A non-primitive function can by definition be written in terms of primitive functions of the same rank (and possibly other functions of smaller rank).  For example, erf is primitive, but erf(erf) is not, although they are of the same rank 1.
Suppose R is the largest rank.  Then the integral f of any rank-R function f' is a non-primitive rank-R function.  It is possible to express f in terms of functions that have a rank smaller than R or are primitive and have rank equal to R, and at least one function of the latter type is required.  The sum of any two primitive functions remains primitive, so in the general case we have such a function appearing in a non-additive expression.  For the same reason we can find a case where it is not the root of the expression tree.  Now we can write f as a composition of a product:
f = g(h * j)
where j is a primitive function of rank R and g and h are some other functions.
The derivatives of f are:
f' = (h * j)' * g'(h * j)
f'' = (h * j)'' * g'(h * j) + (h * j)' * g''(h * j)
f'' contains the same terms that define f' and is therefore also of rank R, so f' is not primitive.  But there must be at least one such primitive function f', so the hypothesis of a largest rank is contradicted.
