Hilbert space homomorphisms and operators When studying Hilbert spaces, the functions I've mostly seen being dealt with is linear functions. Of course, a big part of that is that differential operators are linear and many other interesting examples are too.
However, thinking about mathematical spaces as being "dual" to their set of "structure preserving functions" (I guess the Yoneda lemma makes this rigorous? I'm not strong in category theory), this doesn't sit right with me. Actually, what should the morphisms on  Hilbert spaces be? What I mean is that a bijective morphism from a Hilbert space to itself, with an inverse function that is also a morphism , should be a Hilbert space isomorphism.
Come to think about it, in metric spaces I don't know what the morphisms inducing the "correct" isomorphisms (isometries) should be.
I might have to learn some more category theory besides the basic stuff, and if that's the cause, feel free to point that out :).
 A: It is actually surprisingly complicated to give an answer to this that is even halfway satisfactory; Hilbert spaces are very unusual. First, your question about metric spaces is easier to answer: you can take the morphisms to be short maps, which are maps $f : X \to Y$ satisfying
$$d_Y(f(x_1), f(x_2)) \le d_X(x_1, x_2).$$
This turns out to be a nice and natural choice for several reasons; for more about this you can read about Lawvere metric spaces. But for starters it's clear that an isomorphism in the category of metric spaces and short maps is an isometry (not a homeomorphism as in the usual category of metric spaces and continuous maps, which is better understood as the category of metrizable spaces).
It is also a nice and natural choice for morphisms in the category of Banach spaces; here short linear maps can equivalently be characterized as linear maps with operator norm $\le 1$. The resulting category of Banach spaces has many pleasant features, such as being complete, cocomplete, and closed symmetric monoidal. As two tantalizing hints about how this category works, in this category the coproduct of $S$ many copies of $\mathbb{C}$ is $\ell^1(S)$ (so this is also the free Banach space on $S$), and the product is $\ell^{\infty}(S)$. You can read more about this in my blog post Banach spaces (and Lawvere metrics, and closed categories) which also includes some relevant material about Lawvere metric spaces.
Hilbert spaces are in particular Banach spaces, so we can talk about the category of Hilbert spaces and short maps. But this isn't quite right: among other things, there's no natural way to talk about adjoints in this category (as far as I know). The fix to this is to axiomatize adjoints as a fundamental feature: this leads to the notion of a dagger category, which is a category equipped with a notion of adjoints
$$(-)^{\dagger} : \text{Hom}(a, b) \to \text{Hom}(b, a)$$
satisfying some axioms.
The dagger category of Hilbert spaces and continuous linear maps, together with the dagger structure given by adjoints, recovers everything: the dagger can be used to recover the inner product (this is a nice exercise), and in any dagger category we can talk about unitary maps and hence unitary isomorphisms (as well as self-adjoint maps, skew-adjoint maps, isometries, and other fun stuff like that). We also get the $C^{\ast}$-algebra structure on every endomorphism algebra. You can read a bit more about this in my blog post Hilbert spaces (and dagger categories), although I don't go into a ton of detail on the dagger structure.
Unlike the case of Banach spaces, there is no categorical formalism I'm aware of which has the property that $\ell^2(S)$ is the free Hilbert space on a set $S$, which I'm a little unhappy about.
