Yes.
Observe:
if
$p(x) \in \Bbb R[x], \tag 1$
then given any $\epsilon$ there exists a polynomial
$q(x) \in \Bbb Q[x] \tag 2$
such that
$\vert p(x) - q(x) \vert < \epsilon, \; x \in [a, b]; \tag 3$
for, writing
$p(x) = \displaystyle \sum_0^n p_i x^i, \; p_i \in \Bbb R, \tag 4$
$q(x) = \displaystyle \sum_0^n q_i x^i, \; q_i \in \Bbb Q, \tag 5$
then, assuming $a \ne b$,
$\vert p(x) - q(x) \vert = \vert \displaystyle \sum_0^n p_i x^i - \sum_0^n q_i x^i \vert \le \sum_0^n \vert p_i - q_i \vert \vert x \vert^i \le \sum_0^n \vert p_i - q_i \vert (\max(\vert a \vert, \vert b \vert))^i; \tag 6$
given $p_i$, $0 \le i \le n$, we may choose $q_i$ such that
$\vert p_i - q_i \vert < \dfrac{\epsilon}{(n + 1)(\max_{0 \le i \le n}(\max(\vert a \vert, \vert b \vert))^i)}; \tag 7$
then
$\vert p(x) - q(x) \vert \le \displaystyle \sum_0^n \vert p_i - q_i \vert (\max(\vert a \vert, \vert b \vert))^i$
$< \displaystyle \sum_0^n \dfrac{\epsilon(\max(\vert a \vert, \vert b \vert))^i)}{(n + 1)(\max_{0 \le i \le n}(\max(\vert a \vert, \vert b \vert))^i)}$
$\le \dfrac{\epsilon(\max(\vert a \vert, \vert b \vert))^i)}{(\max_{0 \le i \le n}(\max(\vert a \vert, \vert b \vert))^i)} \le \epsilon. \tag 8$
It follows that for every $\epsilon > 0$ and any $p(x)$ as in (1), there exists $q(x)$ as in (2) with $\vert p(x) - q(x) \vert < \epsilon$ on $[a, b]$. Now replacing $\epsilon$ with $\epsilon / 2$ and choosing $p(x) \in \Bbb R[x]$ such that
$\vert f(x) - p(x) \vert < \dfrac{\epsilon}{2}, \; x \in [a, b], \tag 9$
we may find some $q(x) \in \Bbb Q[x]$ such that
$\vert p(x) - q(x) \vert < \dfrac{\epsilon}{2}, \tag{10}$
and combining (9) and (10) we have
$\vert f(x) - q(x) \vert = \vert f(x) - p(x) + p(x) - q(x) \vert$
$\le \vert f(x) - p(x) \vert + \vert p(x) - q(x) \vert < 2 \dfrac{\epsilon}{2} = \epsilon, \; x \in [a, b], \tag{11}$
or
$\Vert f(x) - q(x) \Vert < \epsilon. \tag{12}$