Given a group (finite or infinite), is there a way to find all its subgroups? I know that for some small groups, one could just brute force all combinations of elements to find its subgroups, but is there some sort of an algorithm, at least for finite groups?
 A: There is a method called the cyclic extension method, which can find the subgroups of a finite soluble group.  (It is possible to enhance this algorithm with knowledge of perfect groups to deal with groups that are insoluble.)  The basic idea is to build up the subgroups in layers, starting with the subgroups of prime order.  To get things rolling, start with the subgroups $\langle g\rangle$ of prime order.  Now you've got layer $1$, and you can proceed inductively.  Once you have a subgroup $H$ in, say, the $i$th layer, a subgroup in the $(i+1)$st layer can be constructed as $\langle H, u\rangle$, where $u$ is an element of the normaliser of $H$ such that $u^p$ belongs to $H$, for some prime $p$.  Putting it all together and making it efficient is rather complicated, but there is a description of it in Chapter 6 of the book "Fundamental Algorithms for Permutation Groups", by Gregory Butler (LNCS #559).  Another, more sophisticated method is described in Section 10.4 of "Handbook of Computational Group Theory", by Holt, Eick and O'Brien (Chapman & Hall/CRC, 2005).
A: This is just to add a couple of comments to James' answer, as applied to finite groups. As he mentioned, to extend the cyclic extension method to non-solvable groups, you need a knowledge of the finite perfect groups, and these are only classified up to some specific order.
In fact all of the current practical methods use by Computer Algebra Systems rely on some kind of database lookup, so they are not complete algorithms. There are effective methods (based on results proved in the 1980s by Aschbacher, Scott, Kovacs and probably others) for reducing the problem to the subgroups of almost simple groups (which are groups $G$ with $S \le G \le {\rm Aut}(S)$ with $S$ nonabelian simple).
For the smaller almost simple groups, it is helpful simply to store (conjugacy class representatives) of their subgroups. This becomes impractical for larger groups, but recursively it is sufficient to store their maximal subgroups, and there is a vast amount of literature about the maximal subgroups of almost simple groups, much of which can be used algorithmically. For example the geometric maximal subgroups of the classical groups (i.e. things like stabilizers of subspaces, systems of imprimitivity) can be constructed generically. There are unfortunately other types of maximal subgroups that have to be classified dimension-by-dimension, but we now have a just about complete description of the maximal subgroups of the classical groups up to dimension 12.
The situation for the alternating groups is similar. I think their maximal subgroups are known and listed up to degree about 4095. Apart from a small number of remaining uncertainties about the Monster, the maximal subgroups of the sporadic groups are completely known and, except for the very large instances, effectively constructible. There is much less known in general about the larger exceptional groups of Lie type.
