I want to point out something potentially misleading about Marek's answer. The n-categories he mentions are not categories, but generalizations of them, so the question still remains, why do categories only form a 2-category, that is, why do people stop after categories, functors, and natural transformations? Why don't people define modifications of natural transformations?
I think it is good to realize that categories really are in an essential way only 2-categorical, if you want interesting higher morphisms you do need to define something like a higher category. One way to think about it is this: natural tranformations are basically homotopies. To make this precise, take I to be the category with two objects, 0 and 1, one morphism from 0 to 1 and the identity morphisms. Then, it is easy to check that to specify a natural tranformation between two functors F and G (both functors C → D) is the same as specifying a functor H : C × I → D which agrees with F on C × {0} and with G on C × {1}.
So then we could get higher morphisms by saying they are homotopies of homotopies, i.e., functors C × I × I with appropriate restrictions. This works, and we indeed get some definition of modification, but it is not interesting as it reduces to just a commuting square of natural transformations, i.e., it can be described simply in terms of the structure we already had.
This is similar to what happens for, say, groups: you can think of a group as a category with a single object where all the morphisms are invertible (the morphisms are the group elements and the composition law is the group product). Then group homomorphisms are simply functors. This makes it sound as if groups now magically have a higher sort of morphisms: natural tranformations between functors! And indeed they do, they are even useful in certain contexts, but they're not terribly interesting: a natural transformation between to group homomorphisms f and g is simply a group element y such that f(x) = y g(x) y -1 . Again, this is described in terms of things we already new about (the group element y and conjugation), and is not really a brand new concept.