The example being the summation of all natural numbers
$$
1+2+3+\cdots=-1/12
$$
where obviously the set of all natural numbers is not closed under infinite addition.
I'll go ahead and jump in, and say here that this summation is just incorrect in the traditional sense for a number of reasons. I explained why here recently - it's a bit long, and I'd rather not deviate from your main focus too much.
That said, proper examples do exist, for example:
$$\sum_{n = 1}^\infty \frac{1}{n^2} = \frac{\pi^2}{6}$$
The rationals $\Bbb Q$ form a field and thus are closed under (finite) addition, but $\pi^2/6$ is irrational, so this is a valid example of what you're trying to get at anyways. You could also use
$$\prod_{p \; prime} \frac{1}{1-p^{-2}} = \frac{\pi^2}{6}$$
as another example for products, for instance. The operand here is a rational number, but there are infinitely many primes, and the product of these infinitely many rationals is irrational. (Similar examples also work for $\zeta(s)$ where $s$ is an even positive integer, but these are the most famous.)
Is the saying correct?
So yes, it is only for finite composition you can say this. Clearly, infinite composition poses problems.
Is the summation of all natural numbers a good counter-example?
As noted above, no, but at least others exist.
Are there other less flimsy examples such as some examples involving finite intersection and infinite intersections?
I think I might have one, but am not totally sure. Anyhow, let's consider
$$A_n = \{ \text{the set of rationals q such that 0 < q < 1/n} \} = (0,1/n) \cap \Bbb Q$$
Obviously, taking finitely many intersections of any of these $A_n$ (indexed by $\Bbb N$), you'll have some sets $A_{n_1},\cdots,A_{n_k}$, and
$$\bigcap_{j=1}^k A_{n_j} = (0,m)\cap \Bbb Q$$
where $m = 1/\max_j \{n_j\}$. Moreover, this intersection is obviously nonempty and corresponds to a different set in the set (namely the one indexed by the maximum). Thus, the set of these $A_n$ is closed under finite composition. However, clearly at the same time
$$\bigcap_{n=1}^\infty A_n = \emptyset$$
which does not correspond to any $A_n$.
Granted, these $A_n$ do not form a group by my understanding - I can't think of how one might assign a group structure to this offhand - but our concern here seems to be composition more than strictly groups. One of the commenters on the question proper mentioned something about topological groups having a more definite notion of infinite composition - I'm far from qualified to talk on that topic at the moment however, having never studied them, so ... sorry in that respect.
