All the answers point to yes, but you need to be careful as what you mean by inverse (of course, mathematics always requires thinking). I will try not to get into set-theoretic issues and appeal to your intuition.
If $f : X \to Y$ is a map of sets which is injective, then for each $x \in X$, we have an element $y = f(x)$ uniquely determined by $x$, so we can define $g : Y \to X$ by sending those $y \in f(X)$ to that element $x$ for which $f(x) = y$, and the fact that $f$ is injective will show that $g$ will be well-defined ; for those $y \in Y \backslash f(X)$, just send them wherever you want (this would require this axiom of choice, but let's not worry about that). The function $g$ satisfies $g(f(x)) = g(y) = x$, so that $g \circ f$ is the identity map ; that is, $f$ admits a left inverse.
Conversely, suppose $f$ admits a left inverse $g$, and assume $f(x_1) = f(x_2)$. Then $x_1 = g(f(x_1)) = g(f(x_2)) = x_2$, so $f$ is injective. That is,
A function $f : X \to Y$ is injective if and only if it admits a left-inverse $g : Y \to X$ such that $g \circ f = \mathrm{id}_X$.
Similarly, it is not hard to show that $f$ is surjective if and only if it has a right inverse, that is, a function $g : Y \to X$ with $f \circ g = \mathrm{id}_Y$. I'll let you ponder on this one.
If you're looking for a little more fun, feel free to look at this ; it is a bit harder though, but again if you don't worry about the foundations of set theory you can still get some good intuition out of it.
Hope that helps,