Why were Lie algebras called infinitesimal groups in the past? And why did mathematicians begin to avoid calling them infinitesimal groups and switch to calling them Lie algebras?
I don't know anything about the actual history, but you might want to look at this blog post by Terry Tao. Terry defines a "local group" to be something that looks like a neighborhood of the identity in a topological group and a "local Lie group" to be something that looks like a neighborhood of the identity in a Lie group.
He then defines a "group germ" to be an equivalence class of local groups, where the equivalence relation should be thought of as "become the same under passing to a sufficiently small open neighborhood of the identity". Thus, the group germ remembers all the information which can be seen on an arbitrarily small neighborhood of the identity. I think this is a very good modern rigorous analogue of the notion of an infinitesimal group. Terry then proves Lie's Third Theorem (Theorem 2 in his notes): Germs of local lie groups are in bijection with Lie algebras.
If you prefer algebra to analysis, the corresponding idea is a formal group. Again, it is a true but nontrivial theorem that formal groups in characteristic zero are equivalent to Lie algebra.
Thomas Hawkins's Emergence of the Theory of Lie Groups: An Essay in the History of Mathematics 1869–1926 (2000) answers your question.