The answer to this is no, in general; it is in fact possible to have $\mathbb{Q}^{\text{alg}}$ as a subfield of a pseudo-finite field of the form you indicate. This follows from the following lemma:
If $f(x)\in\mathbb{Z}[x]$, then there are infinitely many primes $p$ such that $f$ splits completely mod $p$.
For a really slick proof of this fact, see Edit 2 of Qiaochu's answer here. (He says he learned the argument from a MathOverflow post of Bjorn Poonen.)
Okay, to see why this lets us construct the desired ultraproduct: for a polynomial $f\in\mathbb{Z}[x]$, let $P_f$ be the set of primes modulo which $f$ splits completely. For any $f_1,\dots,f_n$, by applying the above lemma to $f=f_1\cdot\ldots\cdot f_n$, we know that $P_f=P_{f_1}\cap \dots \cap P_{f_n}$ is infinite. In particular, if $P$ is the set of all primes, taking the filter on $P$ generated by $(P_f)_{f\in\mathbb{Z}[x]}$ yields a filter $\mathcal{F}$ containing no finite set.
We may thus complete $\mathcal{F}$ to a non-principal ultrafilter, and then taking the ultraproduct of the $(\mathbb{F}_p)_{p\in P}$ along this ultrafilter will yield a characteristic $0$ field in which every polynomial with integral coefficients splits completely, ie a field containing $\mathbb{Q}^{\text{alg}}$, as desired.