There is an infinite group which contains every single finite group! Just take the direct sum of all finite groups...You get a group, and this is infinite. However, this is boring because the group is so wild. For example, there is no finite set which will generate this group (note that the same problem exists for Christian Blatter's group - in fact, his group is uncountable while this one is countable). Often the challenge in group theory is to give finitely presentable groups with wild properties, such as containing all finite groups. For example, in the group we have just constructed every element has finite order, but it is an open problem if there exists a finitely presentable group in which every element has finite order.
In order to overcome this problem of non-finite generation/presentation we need quite a big theorem, called Higman's Embedding Theorem. This states that a group is "recursively presentable" if and only if it can be embedded into a finitely presentable group. Rotman's book An introduction to the theory of groups contains an accessible proof of Higman's Embedding Theorem.
Thereom: There is a finitely presented group which contains every single finite group. Specifically, this group is finitely generated and countable.
Proof: Start by enumerating all finite groups and then take their direct sum using this enumerations. (Basically, do what I said in the first paragraph but be careful when doing it). This gives you a recursively presented group, and so you can use Higman's Embedding Theorem to obtain a finitely presented group containing your group. Done.
(Note that the above proof actually works for all finitely presented groups but not all countable groups...but to do that properly I'd have to convince you that there are only countably many countable groups but uncountably many uncountable groups. These facts both hold, but are stories for a different day...)