Regarding the question about what type of sheaves are under consideration, the answer is coherent sheaves. The corresponding algebra concepts are from the theory of finitely generated modules over Noetherian rings (and especially, from the theory of depth and related concepts).
As noted in comments, the property of reflexivity implying local freeness is special to surfaces. On a variety, a reflexive sheaf will be $S_2$ (where $S_n$ are the depth conditions that appear in the definition of Cohen--Macaulay: on a $n$-dimensional variety, being Cohen--Macaulay is the same as being $S_n$).
A theorem of Auslander--Buchsbaum shows that on a smooth variety, a coherent sheaf that is Cohen--Macaulay is locally free. So for smooth surfaces, reflexive equals $S_2$ equals Cohen--Macaulay equals locally free.
If $X$ is a smooth three-fold, if $\mathcal I$ is the ideal sheaf of a point, and if $\mathcal K$ denotes the kernel of a surjection $\mathcal F \to \mathcal I$, with $\mathcal F$ locally free, then $\mathcal K$ is reflexive, i.e. $S_2$, but not locally free.
Incidentally, torsion free is the same as $S_1$, and can be thought of geometrically as saying that the sheaf admits no non-zero sections whose support is a proper closed subvariety. (More generally, if you are on a variety or scheme that is allowed to be reducible, then the condition is that the support of a non-zero section should be a union of irreducible components.)
Regarding the use of rank in the non-locally free context:
the generic fibre of a torsion free coherent sheaf is a finite rank vector space over the function field of $X$, and this is how its rank is defined.
Finally: I've found these notes of Karl Schwede helpful in the past when trying to think about this material.