What separates rotations from other co-ordinate transformations? I am confused about some seemingly elementary ideas. From what I have understood, a rotation is just a specific class of co-ordinate transformations. If this is true, what exactly separates a rotation from the other transformations? I will give an example below. 
Let us take an $n$-dimensional manifold (we have provided with a metric), to label points on the manifold we use have co-ordinates  $$x^1,...,x^n$$
Let us transform the co-ordinates into new co-ordinates $$x'^k=x'^k(x^1,...,x^n) $$ where $k$ goes from $1$ to $n$. These functions are single valued, continuous and have continuous derivatives.
The elements of the metric (for the original co-ordinate system) are $f_{ij}(x_1,...,x_n)$. The elements of the new metric $f'_{ij}$ are related to the old as $$f_{ij} = f'_{kl}{a^{k}_{i}}{a^{l}_{j}}$$ where $a^{r}_{s}$ is an element of the Jacobian.
Now, what exactly are the restrictions that must be imposed on the elements ${a^{r}_{s}}$ to constitute a 'rotation'? I might have omitted some details, so you are welcome to make reasonable assumptions.
 A: Part of your confusion stems from not distinguishing between position vectors (or coordinate tuples) and tangent vectors.  A parameterized curve through space that is sufficiently smooth and differentiable has a tangent vector as its derivative at each point.
The reason for this distinction is practical: a coordinate transformation can change position vectors in a general, nonlinear way.  Tangent vectors, however, always transform in a linear way, according to the Jacobian of the nonlinear transformation.  The nature of these transformations emphasizes a generalized notion of translational invariance:  rather, the freedom to choose coordinates for any problem.
That's how position vectors and tangent vectors differ in how they transform with respect to changes in coordinates.  There's a second kind of transformation, however, that doesn't involve any change of coordinates at all.  Tangent vectors can be rotated globally according to some arbitrary, position-dependent function, in a way that involves no changes of coordinates at all.  This corresponds to a generalized notion of rotational invariance.  Relations between tangent vectors need not be made with reference to any particular direction, only between each other.
It's important to understand that the second notion, that of rotational invariance, does not rely upon the coordinate system invariance I alluded to earlier.  Rotating a whole coordinate system is entirely different from rotating only tangent vectors.
Thus, while rotation can be a coordinate system transformation, it isn't always.
A: In Euclidean space $\mathbb R^n$, rotations (which together make up the special orthogonal group $SO(n)$) are the linear transformations that preserve oriented orthonormal bases. If you use the usual orthonormal basis then these are exactly the linear transformations whose matrices satisfy $A^TA = I$ and $\det A = +1$.
Since the spheres $\mathbb S^{n-1}$ are embedded in $\mathbb R^n$ and are preserved by rotations, this also gives us the definition of a rotation on a sphere. You can find qualitatively similar isometries of hyperbolic space.
In the Euclidean setting, the definition I gave above distinguishes the origin - Iactually defined the rotations about the origin. This is a natural notion when looking at $\mathbb R^n$ as a normed vector space, but when we forget everything except the Riemannian structure, the origin is not at all special. We can generalise this to rotations about arbitrary points by allowing conjugation by translations (i.e. define rotations to be $BAB^{-1}$ where $A$ is a rotation about the origin and $B$ is a translation).
The most common generalization to arbitrary Riemannian manifolds that I know of is quite a restrictive one (that is, very few manifolds will satisfy this): the notion of rotational symmetry. We say an $n$-manifold $(M,g)$ has rotational symmetry about $p$ if the isometry group of $(M,g)$ contains a copy of $SO(n)$ fixing $p$. (When the underlying topological manifold is $\mathbb R^n$ these metrics are familiar as those that can be written $g= dr^2 + \rho^2(r) d \Omega^2$.) The isometries belonging to this subgroup are then referred to as rotations about $p$. Similarly I would say it reasonable to call any action of $SO(k)$ by isometries a group of rotations.
Note that this definition of a rotation is not in terms of the properties of a single isometry, but rather of a group of them. I'm not sure if you can make a reasonable characterisation of rotations in terms of the individual transformations - the only distinguishing properties I can think of seems to be orientation preservation.
A: In the three-dimensional case, a rotation matrix is one whose rows (or columns) are orthogonal unit vectors, and whose determinant is $+1$. Geometrically, this means as follows: the orthogonality means there is no "shearing", the unit lengths mean there is no stretching, and determinant $= +1$ (as opposed to $-1$) means there is no mirroring/reflection. Those properties are what you would expect for a transformation that claims to be a "rotation". 
I expect all of this can be generalized to the more general situation you asked about, but developing some intuition about the 3D case first is a good start.
