What problems arise when using infinitesimals in calculus? In contemporary real analysis we use a limit definition in terms of deltas and epsilons.
Before that, people used infinitesimals to calculate limits.
Is there a specific non-philosophical reason why we didn't keep on using infinitesimals?
I.e. are there concrete examples in which the use of infinitesimals lead to serious problems?
 A: Before the formalization of limit in terms of $\epsilon $ and $\delta $ the arguments given in analysis were heuristic, simply because at the time no known model of reals with infinitesimals was known. People used infinitesimals intuitively, though they knew no infinitesimals existed (at least for them, at the time). The fact that (correct, in whatever sense) use of infinitesimals did not lead to any blunders was somewhat of a strange phenomenon then. Once Cauchy formalized limits using $\epsilon $ and $\delta $ it became possible to eliminate any infinitesimals from the formal proofs. One could still think infinitesimally, or not, but one could finally give rigorous proofs. 
Things changed when Robinson discovered a construction, using tools from logic that were new at the time, by which one can enlarge the reals to include actual infinitesimals. Retrospectively, this discovery explained why infinitesimals did not lead to blunders. Simply since they do exist! 
Today inertia dictates one's first encounter with analysis, and so non-standard analysis is usually never met until one stumbles upon it or in advanced courses, usually in logic rather than analysis. Having said that, there are textbooks aimed at a beginner's course in calculus using non-standard analysis. There are probably two reasons why that is unlikely to catch momentum. First is the name; nobody really wants to do things non-standardly. Secondly, and more importantly, the prerequisites for Cauchy's $\epsilon $ $\delta $ formalism is very modest. However, even the simplest models of non-standard analysis require a significant dose of logic, one that will take a week or two at least of a beginner's course. And since non-standard analysis is as powerful as ordinary analysis, it is difficult to justify putting in the logic(al) effort, for what many may consider to be only cosmetic gain. Some, disagree though and claim non-standard analysis is superior.  
A: The objections to infinitesimals were metaphysical. The Volume 14 of the Encyclopædia Britannica 1911 says: 

The name "infinitesimal" has been applied to the calculus because
  most of the leading results were first obtained by means of arguments
  about "infinitely small" quantities; the "infinitely small" or
  "infinitesimal" quantities were vaguely conceived as being neither
  zero nor finite but in some intermediate, nascent or evanescent,
  state. There was no necessity for this confused conception, and it
  came to be understood that it can be dispensed with; but the calculus
  was not developed by its first founders in accordance with logical
  principles from precisely defined notions, and it gained adherents
  rather through the impressiveness and variety of the results that
  could be obtained by using it than through the cogency of the
  arguments by which it was established.

Initially, mathematicians considered negative numbers to be "absurd". Later, complex numbers were considered "absurd" too. In 1911, no mathematician considered negative or complex numbers as absurds. They were already accepted. Surprisingly, infinitesimals were considered to be "absurd"!
How come people could accept things like $\sqrt{-1}$ but could not accept infinitesimals? I think the answer is because of prejudice against them!
Archimedes was the first to use infinitesimals. He considered that they lacked rigor, and then, although he often used them to obtain his results, when he published proofs of these results, he never mentioned infinitesimals! Instead, he used the method of exhaustion.
A: This is intended as both an answer and a comment. @Conifold, The idea that "the way Leibniz and others were manipulating infinitesimals to find derivatives, for example, led to contradictions" is a misconception. There is a number of recent articles on this in journals ranging from Notices AMS to Erkenntnis. The articles are a bit long but if you are looking for a concise summary you could consult this review by Marcel Guillaume.
