When the homsets of a category are structured. If the objects of a category are algebraic structures in their own right, this often places additional structure on the homsets. Is there somewhere I can learn more about this general idea?

Example 1. Let $\cal M$ denote the category of magmas and functions, not necessarily structure-preserving. Then given objects $X$ and $Y$ of $\cal M$, the hom"set" $\mathrm{Hom}(X,Y)$ is probably best viewed as a magma as opposed to a set, with the pointwise operation inherited from $Y$. Furthermore we have a right (but not left) distributivity law. In particular, given objects $X, Y$ and $Z$ and arrows $f:X \rightarrow Y$ and $g,g' : Y \rightarrow Z,$
$$(g+g')\circ f = (g \circ f) + (g' \circ f).$$
Example 2. Now let $\cal N$ denote the category of medial magmas and magma homomorphisms. Then given objects $X$ and $Y$ of $\cal N$, the hom"set" $\mathrm{Hom}(X,Y)$ can be viewed as a medial magma in its own right. Furthermore, we have both left and right distributivity laws. In the sense that given objects $X, Y$ and $Z$ and arrows $f,f':X \rightarrow Y$ and $g,g' : Y \rightarrow Z,$
$$(g+g')\circ f = (g \circ f) + (g' \circ f)$$
$$g \circ (f + f') = (g \circ f) + (g \circ f').$$
 A: You are looking for the notion of an enriched category. A good reference is Kelly's book. For example, the category of medial magmas is monoidal and closed, hence enriched over itsself. Your example of magmas doesn't fit to this picture, basically because it is a little bit unnatural to consider all functions as morphisms.
A: This will be my humble attempt to answer "why enriched, why monoidal?"
So in a general category, we can just view the $Hom$-objects as being sets, the monoidal operation being the Cartesian product, and the "unit" is any terminal object. It seems trivial to note, but the composition operation out of $Hom(A,B)\times Hom(B,C)$ is a morphism in the category of sets, while the Cartesian product is also a set.
This observation becomes less trivial when we move to, say, Abelian groups or categories as $Hom$-objects. What we have in, say, $\mathbf{Cat}$ is not just a category structure on the $Hom$ objects, but the composition operation that maps $B^A\times C^B$ to $C^A$ is itself functorial. The trivial category is obviously a unit for the product operation in $\mathbf{Cat}$, and functors from it can pick out the identity functor in any category $A^A$, which is the endofunctor of $A$ that acts as a unit in the composition operation.
That's the point about monoidal categories. There are lots of arbitrary structures you could add to a $Hom$-set, but not every category's composition operation will respect that structure in an appropriate way. Further, not every category has the right structure for its objects to be the $Hom$-objects of a category. If you diagram out how the composition operation works in $\mathbf{Set}$, then you can see it's exactly the kind of structure that a category's being monoidal makes possible. And enrichment is the condition that composition respects the additional $Hom$ structure in the appropriate way.
