This may not be the easiest method, but you appear to be interested in a contour way of going about it.
starting from your $$-a/b\int_{0}^{\infty}\frac{\sin(bx)}{e^{ax}-1}dx$$
Consider the function$$f(z)=\frac{e^{ibz}}{e^{az}-1}$$
Use a rectangle in the first quadrant with height $\displaystyle 2\pi i/a$ with quarter circle indents around $2\pi i/a$ and $0$.
There will be 6 portions to put together:
$$I_{1}+I_{2}+I_{3}+I_{4}+I_{5}+I_{6}=0........(1)$$
The integral can be set to 0 because there are no poles inside the contour.
Along bottom horizontal on x axis: $$I_{1}=\int_{\epsilon}^{R}\frac{e^{ibx}}{e^{ax}-1}dx$$
up right vertical side:
$$\left|\frac{e^{ibR}}{e^{aR}-1}\right|\to 0, \;\ as \;\ R\to \infty$$
$$I_{2}=0$$
along top horizontal: $$I_{3}=-\int_{\epsilon}^{R}\frac{e^{ib(x+2\pi i/a)}}{e^{a(x+2\pi i/a)}-1}dx=-e^{-2\pi b/a}\int_{\epsilon}^{r}\frac{e^{ibx}}{e^{ax}-1}dx$$
top quarter circle around indent at $2\pi i/a$,
where x varies from
$(\epsilon, \epsilon+\frac{2\pi i}{a})$ to $(0,\frac{2\pi i}{a}-\frac{2\pi i}{a}\epsilon)$
$$I_{4}=\frac{-\pi i}{2}Res\left(f(z), \frac{2\pi a}{b}\right)=\frac{-\pi i}{2}\cdot \frac{e^{ib(2\pi i/a)}}{ae^{a(2\pi i/a)}}=\frac{-\pi i}{2a}e^{-2\pi b/a}$$
Down left vertical side. parameterize with $\displaystyle z=iy, \;\ dz=idy$
$$I_{5}=-i\int_{\epsilon}^{2\pi/a}\frac{e^{-by}}{e^{ayi}-1}dy$$
Quarter circle indent around the origin with x varying from $\displaystyle (0,i\epsilon)$ to $\displaystyle (\epsilon, 0)$.
$$I_{6}=\frac{-\pi i}{2}Res(f,0)=\frac{-\pi i}{2}\cdot \frac{e^{ib(0)}}{ae^{a(0)}}=\frac{-\pi i}{2a}$$
Now, assemble all the portions by plugging them all into (1):, and let $\displaystyle \epsilon\to 0, \;\ R\to \infty$
$$\int_{C}\frac{e^{ibz}}{e^{az}-1}dz=\int_{0}^{\infty}\frac{e^{ibx}}{e^{ax}-1}dx+I_{2}-e^{-2\pi b/a}\int_{0}^{\infty}\frac{e^{ibx}}{e^{ax}-1}dx$$
$$-\frac{\pi i}{2a}e^{-2\pi b/a}-\frac{\pi i}{2a}-i\int_{0}^{2\pi /a}\frac{e^{-by}}{e^{ayi}-1}dy=0$$
$$\rightarrow (1-e^{-2\pi b/a})\int_{0}^{\infty}\frac{\sin(bx)}{e^{ax}-1}dx+\int_{0}^{2\pi/a}\frac{(-i)e^{-by}}{e^{ayi}-1}dy=\frac{\pi i}{2a}(1+e^{-2\pi b/a})$$
By taking imaginary parts, the last integral(the one going down the left vertical side) can be shown to be equal to
$$\int_{0}^{2\pi/a}\frac{e^{-by}}{2}dy=\frac{1-e^{-2\pi b/a}}{2b}$$
solving for the integral in question, we finally have:
$$\int_{0}^{\infty}\frac{\sin(bx)}{e^{ax}-1}dx=\frac{\frac{\pi}{2a}(1+e^{-2\pi b/a})-\frac{1-e^{-2\pi b/a}}{2b}}{1-e^{-2\pi b/a}}$$
$$=\frac{\pi}{2a}\coth(\frac{\pi b}{a})-\frac{1}{2b}$$
multiplying this by the $-a/b$ from the beginning reduces it to a form that can be written in terms of hyperbolic trig functions as the solution suggests.
and, we ultimately get:
$$\frac{a}{2b^{2}}-\frac{\pi}{2b}\coth(\frac{\pi b}{a})$$