Without more information about $f$, other than bounded and integrable, we can't even be sure that $\xi$ exists.
However, as you may have seen, if $f$ has the intermediate value property (IVP), then there exists at least one $\xi \in [a,b]$ (and possibly more) such that
$$\tag{*} \int_a^b f \, d\alpha=f(\xi)[\alpha(b) - \alpha(a)]$$
For example, $f$ has the IVP if continuous or if (by Darboux's theorem) there is an antiderivative, $f(x) = F'(x)$.
Must there exist $\xi \in (a,b)$, in the open interval, such that (*) holds?
The answer is affirmative.
Defining $m= \inf_{x\in[a,b]} \, f(x)$ and $M = \sup_{x\in[a,b]} \, f(x) $, we know that
$$m \leqslant k =\frac{\int_a^b f \, d\alpha}{\alpha(b) - \alpha(a)} \leqslant M.$$
First, suppose we have strict inequality, $m < k < M.$ By the properties of $\inf$ and $\sup$, there exist $c,d \in [a,b]$ such that $m < f(c) < k < f(d) < M$. Since $f$ has the IVP, there exists $\xi \in (c,d) \subset [a,b]$ such that $f(\xi) = k$. Note that $\xi$ is in the open interval $(c,d)$ since $f(\xi) \neq f(c),f(d)$.
On the other hand, suppose that $k = m$. (The case where $k = M$ is handled in a similar way).
Since $f(x) \geqslant m$ and $\alpha$ is increasing, we have
$$\int_a^b |f(x) - m| \, d\alpha = \int_a^b (f(x) - m) \, d\alpha = (k -m)[\alpha(b) - \alpha(a)] = 0,$$
and it follows that $f(x) = m$ almost everywhere and there exists $\xi \in (a,b)$ such that $f(\xi) = m = k$.