Does $\mathbb{Z}$ with subtraction as morphisms, forms a category? Let object of the category be integers and for objects $a$ and $b$, define:
$$
\operatorname{hom}\left(a, b\right) = \left\{ f \in \mathbb{Z} | a - f = b\right\} 
$$
Then composition of $f \in \operatorname{hom} \left(a,b\right)$ and $g \in \operatorname{hom} \left(b,c\right)$ is addition which is associative. Also for all integers $x$, $ \exists 0 \in \operatorname{hom}\left(x,x\right)$ which acts like an identity, but then obviously all objects have the same identity. Does this prevent us from considering $\mathbb{Z}$ with subtraction as a category?
 A: I suggest to change to
$\operatorname{hom}(a, b) = \{ f \in \mathbb{Z} | a=b+f\}$,
this will facilitate generalization.

then obviously all objects have the same identity. Does this prevent
  us from considering Z with subtraction as a category?

It depends on what formal definition of a category you choose. If there it is required that each morphism has a unique source object and a unique target object, then we define a category in the other way: a morphism is $(a, f, b)$ such that $a=b+f$.
You can generalize this definition to any monoid $M$ and further to any action $T$ of a monoid $M$ on a set $X$. Objects are $X$, the condition for $f$ to be a morphism is $a=T(f)(b)$. It is called the transport category of $T$.
A: In Categories, Allegories, Freyd and Scedrov give a way of formally presenting a category that addresses this.
The idea is to present the category in terms of objects and proto-morphisms, and a ternary relation of the form "the proto-morphism $f$ can be construed as going from $A$ to $B$". So the arrows of a category so presented is the graph of this relation -- i.e. the class of all triples $(A,f,B)$ that satisfy the ternary relation.
Furthermore, composition is phrased in terms of proto-morphisms, rather than morphisms. (it is possible but rare for composition of proto-morphisms to be multi-valued -- but it has to be single valued, of course, in the context of the product of actual arrows)
Freyd, Scedrov then list a bunch of axioms that I won't list here; they're just the category axioms stated in these terms in a diagrammatic language.
The point is, $4$ and $7$ in your example are proto-morphisms; we can see that $4+7=11$, but we haven't actually talked about arrows yet. However, at our leisure we can consider $4$ as a morphism from $13$ to $17$, and then we're talking about an actual arrow.

This sort of thing is only important in an arrow-oriented formulation of categories. In an object-and-homset formulation, it doesn't matter if multiple homsets overlap; it's perfectly fine for the identity map to be the same for every object. Heck, it's fine if the entire category only has a single arrow!
An example of why this is fine is composition: it is not a (partial) operation on arrows: it is a family of functions $\circ_{xyz} : \hom(y,z) \times \hom(x,y) \to \hom(x,z)$. The math doesn't get "confused" by arrows in different homsets being equal, because by using $f \circ_{xyz} g$, we "remember" that $f$ goes from $y$ to $z$ and $g$ goes from $x$ to $y$.
A: No. One can define a category on vertex set $X$ using $\hom:X\times X\to Set$ and associative composition, without the assumption that $\hom(a,b)$ and $\hom(x,y)$ need be disjoint..
However, the category you obtain, as Hagen also wrote, contains exactly one arrow $a\to b$ for any pairs $(a,b)$, and hence every arrow is invertible (i.e., it is a grupoid).
The same construction applies on any set H: let $\hom(x,y):=\{(x,y)\}$, i.e. consider the full graph (with loops) on $H$..
