Consider the following slightly more explicit definition:
Suppose $f$ and $\alpha$ are bounded real-valued functions on $[a,b]$ and we have a partition $P=\{x_0,\dotsc,x_n\}$ of $[a,b]$ and set of tags $T = \{t_1,\dotsc,t_n\}$ with $t_i \in [x_{i-1},x_i]$. Then we call
$\qquad \displaystyle S_\alpha(f,P,T) := \sum_{i=1}^n f(t_i) \bigl[\alpha(x_i)-\alpha(x_{i-1})\bigr]$
a Riemann–Stieltjes sum for $f$ with respect to $\alpha$.
We say that $f$ is Riemann–Stieltjes integrable with respect to $\alpha$ and write $f \in \mathcal R_\alpha[a,b]$ when there exists a number $I \in \mathbb R$ such that for every $\epsilon > 0$ there is a partition $P_\epsilon$ such for for all finer partitions $P \supseteq P_\epsilon$ and all choices of tags $T$ compatible with $P$:
$\qquad \displaystyle |S_\alpha(f,P,T)-I|<\epsilon$
and in that case we write $\int_a^b f\,d\alpha = I$; or $(RS)\int_a^b f\,d\alpha=I$ if we want to emphasize that we're referring to the Riemann–Stieltjes integral.
You can easily convince yourself that the above is essentially a restatement of the definition from the Wikipedia page. It's also a slight rephrasing of a passage from Neal L. Carothers - Real Analysis, Cambridge University Press, 2000. Google books (incomplete), Amazon.
In that section, "Integrators of Bounded Variation", he also remarks that for $\alpha$ increasing we have for all partitions that $L(f,P,\alpha) \leq S_\alpha(f,P,T) \leq U(f,P,\alpha)$. In particular it's easy to show that the integral against an increasing function $\alpha$ defined using upper and lower sums (also called the Darboux–Stieltjes integral, written $(DS)\int_a^b f\,d\alpha$ if one wants to be explicit) agrees with the Riemann–Stieltjes integral.
But upper and lower sums don't necessarily make sense when $\alpha$ is only assumed to be of bounded variation. However Carothers has as Exercise 39 in Chapter 14 to prove that one can compute the Riemann–Stieltjes integral $\int_a^b f\,d\alpha$ using the Jordan decomposition $\alpha = p-n$ as the difference of two Darboux–Stieltjes integrals, i.e.
$\qquad\displaystyle (RS)\!\int_a^b f\,d\alpha = (RS)\!\int_a^b f\,dp - (RS)\!\int_a^b f\,dn = (DS)\!\int_a^b f\,dp - (DS)\!\int_a^b f\,dn$,
where the first equality is just linearity in the integrator of the Riemann–Stieltjes integral and the second equality is the one that requires a bit of work.