Injective implies divisible: generalizing proof from finitely generated to arbitrary abelian group

I am working on showing that in the category of abelian groups, an injective group is divisible (i.e. for each $m\in \mathbb{N}$, the map $a\to ma$ is surjective). Originally I had this thought:

Let $A$ be an injective abelian group. For some $k$, there is a surjection $\mathbb{Z}^k\to A$. Now $\mathbb{Z}^k$ sits inside $\mathbb{Q}^k$, giving rise (by injectivity of $A$) to a surjection $\mathbb{Q}^k\to A$. Now given any $a$ in $A$ and $m\in \mathbb{N}$, choose $\tfrac{p}{q}$ in the preimage of $a$, and consider the image of $\tfrac{p}{mq}$ to show the divisibility.

I then realized that this of course only works if $A$ is finitely generated. My question is, if I consider abelian groups as $\mathbb{Z}$ modules, replace the role of $\mathbb{Z}^k$ above with just some free $\mathbb{Z}$-module $F$, and localize $F$ at $\mathbb{Z}\setminus{0}$ (considering it a $\mathbb{Z}$ module via the inclusion $\mathbb{Z}\hookrightarrow\mathbb{Q}$), does the rest of the proof follow in the same way?

It seems to me that there are no obvious holes in the argument, but I am not yet too comfortable with localizations, and I always get a little (unnecessarily) paranoid when things aren't finitely generated.

Your argument works perfectly. The key facts you need are that if $F$ is a free $\mathbb{Z}$-module then the map $F\to S^{-1}F$ is injective and $S^{-1}F$ is divisible, where $S=\mathbb{Z}\setminus\{0\}$. Divisibility of $S^{-1}F$ is immediate (since it is a module over $S^{-1}\mathbb{Z}=\mathbb{Q}$). Injectivity of the map $F\to S^{-1}F$ follows from the fact that $F$ is torsion-free: in general, if $M$ is any module, the kernel of the map $M\to S^{-1}M$ consists of those $m\in M$ such that $sm=0$ for some $s\in S$ (this follows from the explicit description of $S^{-1}M$ as a set of fractions, since $\frac{m}{1}=\frac{0}{1}$ iff $s\cdot m\cdot 1=s\cdot 0\cdot 1$ for some $s\in S$).
Note that there is a simpler argument you can give, though. To show that $a\mapsto ma$ is surjective on $A$, you just need to show that its image contains any arbitrary element of $A$. So you can fix $a\in A$, and consider the homomorphism $\mathbb{Z}\to A$ sending $1$ to $a$. This then extends to $\mathbb{Q}$, and the image of $\frac{1}{m}$ is an element $b$ such that $mb=a$.