The 4x4x4 cube and higher aren't groups in the same sense that the 3x3x3 cube is a group.
The set of reachable positions of a 3x3x3 cube, viewed as functions from a 54-element set (representing the locations of the stickers) to a 6-element set (the colors of the stickers) form a group. The operation here is given by the following: For x, y positions of a 3x3x3 cube, let $a_1a_2 \cdots a_n$ be a sequence of moves which, starting from the identity, puts the cube into position x, and $b_1b_2 \cdots b_m$ the same for y. The product xy is the state the cube is in after the sequence $a_1 \cdots a_n b_1 \cdots b_m$.
The fact that this operation indeed forms a group isn't as trivial as it first seems. The issue isn't with associativity, identity, or invertibility. Instead, it's with well-definedness. How do you know that the choice of sequences $a_1 \cdots a_n$ and $b_1 \cdots b_m$ doesn't make a difference?
For the 3x3x3 cube, the way to solve this is viewing it as a subgroup of the symmetric group on the set of all 54 stickers. This doesn't work for larger cubes, because it's possible to come up with moves that move some of the cubies around, without changing how the stickers. The stickers can move, without their apparent colors changing. To see why this is impossible for a 3x3x3 cube, note that any cubie is uniquely specified by its stickers, so any permutation of 2 stickers of the same color in the cube group must permute the cubies, which then can not be the identity.
But on the 4x4x4 cube, this fails. All 4 of the starred stickers here, for example, are indistinguishable, in the sense that you could permute them and still have a solved state.
I don't think a permutation of just these 4 stickers is possible, but there are many permutations of indistinguishable cubies which can be done. According to Dustan Levenstein in the comments, any even permutation of these 4 stickers (or more generally of any center stickers) is possible.
The way to prove formally that the 4x4x4 cube is not a group is to find a sequence of moves that acts as the identity on one configuration, but not on another configuration. This is pretty easy to do, but I don't remember the precise solution, so I'll omit it. (If anyone really wants such an example, comment and I can dig one up.)
It is true that if we labeled all of the stickers somehow, forming a so-called "supercube", and required that the labels also match up, then we would have a group. This group would be constructed in the same way as the 3x3x3 group, as a subgroup of the symmetric group on the 96 stickers of the 4x4x4 cube.
This group acts on the set of positions of the 4x4x4 cube transitively, but not freely. This, in group theoretical language, is why the 4x4x4 cube positions do not form a group. We can still study the larger group, but we need to take into account that the action isn't as nice as in the 3x3x3 case.