"What this implies (for me) is the boring consequence that there is no new conceptual richness in higher dimensional geometry beyond the fact than the numbers are larger."
This turns out to be incorrect and should serve as a warning that intuition can lead us astray.
One of the best examples of a completely new property that emerges in higher dimensions involves optimal sphere packing. It transpires that dimensions 8 and 24 are special and different from other dimensions. There exists a lattice in dimensions 8 and 24 which specifies an optimal universal packing:
It also turns out that dimension 4 has the unique property that orbits in 4-D space with 3 spatial dimensions are stable, while in higher-dimensional space they are not.
"In four dimensions, stable orbits are possible. For example, the Earth moves around the sun in the stable orbit, the moon around the Earth in the stable orbit and these are sorts of planar orbits. In higher dimensions, achieving stable orbits is very difficult and generally stable planet orbits don’t exist. And so, it is interesting that somehow Einstein’s equation is telling us that four dimensions are actually the right number of dimensions for life to exists, given that life really depends on the planet moving around the sun and so forth. Four dimensions is really a Goldilocks number. It’s not so too constrained like three dimensions where effectively there is no freedom and no dynamics, but it’s also not too free so that you just have complete chaos, many different solutions, no stable orbits and so on."
Mathematically, certain dimensions associated with symmetry groups have unique properties not found in other numbers of dimensions. For example, the symmetry group E8 (quite famous among mathematicians) describes a space of 248 dimensions. The properties of the E8 Lie Group are unique. More here:
The group of Clifford algebras has various special properties not found in groups of other size (i.e., dimension):
Since a given dimension number is generally dual to a Galois group, groups of different size (i.e., dimension) exhibit different mathematical properties. It turns out the group S6 is the only one with a non-trivial outer automorphism, which makes it unique among groups.
Topology and groups share a close connection, as do the solution spaces of differential equations and abstract algebras. Different numbers of dimensions have special properties, depending on the abstract algebra or differential equation or topological manifold concerned. In other words, certain dimensions like 4 or 8 or 24 or 6 exhibit unique properties depending on the situation in the same way that a specific card like an ace can exhibit a special property in the context of the rules of poker.
Many of the answers above gives examples of the unique behavior of various numbers of dimensions, and there exist many more examples throughout mathematics and physics and chemistry and dynamical systems.