The answer you probably don't want is that these kinds of "implementation details" really don't matter at all. Pick whichever makes you happiest, and it will never impact the answers to category-theoretic questions.
First, let me give a simpler example to make sure that I understand what you're asking:
We have "two possible implementations" for a group homomorphism $(G, \cdot) \to (H, \star)$.
- A triple $((G,\cdot),(H,\star),f)$ where $f : G \to H$ is a map on underlying sets (satisfying certain conditions)
- A function $f : G \to H$ on the underlying sets (satisfying certain conditions)
Now you worry that, under definition $2$, two different homomorphisms can have the same implementation. For instance, if we define $\cdot^\text{op} : G \times G \to G$ as $a \cdot^\text{op} b = b \cdot a$, then the same function $f$ can represent both a homomorphism $(G, \cdot) \to (H, \star)$ and to $(G,\cdot^\text{op}) \to (H,\star^\text{op})$ because these two homomorphisms have the same underlying function!
In definition $1$, we explicitly "tag" our function with its domain and codomain as groups so that the two homomorphisms $((G,\cdot), \ (H,\star), \ f)$ and $((G,\cdot^\text{op}), \ (H, \star^\text{op}), \ f)$ are encoded by different objects in the ambient set theory.
If I understand correctly, you're asking basically the same question, but in the context of categories and functors rather than groups and homomorphisms.
Now, in the group theoretic case and the category theoretic case, these implementation details don't matter. You will never be presented with a naked function out in the wild (or pair of functions in the category case) and asked "what is the group structure on the domain of this function?". Ironically, given the question, we can make this precise using category theory! Whenever we interact with a homomorphism $f$, we're really interacting with $f \in \text{Hom}((G,\cdot), (H,\star))$! We always have the bonus context on hand of what the domain and codomain should be. So, if you want${}^1$, you can absolutely use definition $1$ and tag your homomorphisms (functors) with the group structure (category structure) on their source and target.
Why does category theory make this precise? Because we can define categories of groups $\mathsf{Grp}_1$ and $\mathsf{Grp}_2$ using the two implementations of homsets. It's an easy exercise to show that these categories are equivalent (in fact, isomorphic!) so that, in this precise sense, you cannot distinguish the two definitions using only group theoretic questions. Similarly, you can build two categories $\mathsf{Cat}_1$ and $\mathsf{Cat}_2$ using the two definitions of functor, and you can show that these categories are also isomorphic${}^2$! So the answer to any "category theoretic question" you can ask won't depend on whether you choose definition 1 or 2!
I hope this helps ^_^
${}^1$: and it seems like you do want this, since you say you're doing this for your arrows in $\mathsf{Set}$
${}^2$: you do have to eat your tail a little bit since checking isomorphism of categories requires working in a category of categories. And in this category of categories you need a notion of functor, so you have to choose a definition before you can prove the two definitions agree... But this is really no different than any other time we have two equivalent definitions and we have to choose one "official" one to write down first.