I should rewirte this answer.
Lemma
Let $\mathfrak{p}$ be a prime ideal of $\mathbb{Q}[x_1,\ldots,x_n]$. Let $F/\mathbb{Q}$ be a finite Galois extension. Let $Gal(F/\mathbb{Q})=\{\sigma_1,\ldots,\sigma_f\}$. Let $S=\{\mathfrak{p}_i\}$ be the set of prime ideals of $F[x_1,\ldots,x_n]$ which are lying over $\mathfrak{p}$. Then $G=Gal(F/\mathbb{Q})$ may act transitively on $S$. In particular, $S$ is finite.
Proof
First, $G$ acts on $S$. Suppose $\sigma\in G$. Then $\sigma(\mathfrak{p}_i)\cap \mathbb{Q}[x_1,\ldots,x_n]\supset \mathfrak{p}(=\sigma(\mathfrak{p}))$ and $\sigma^{-1}(\sigma(\mathfrak{p}_i)\cap \mathbb{Q}[x_1,\ldots,x_n])\subset \mathfrak{p}_i\cap \mathbb{Q}[x_1,\ldots,x_n]=\mathfrak{p}$, hence $G$ acts on $S$.
Second, $G$ acts transitively on $S$. Let $\mathfrak{p}_1,\mathfrak{p}_2\in S$ and $x\in \mathfrak{p}_1$. Then $\sigma_1(x)\sigma_2(x)\cdots \sigma_f(x)\in \mathfrak{p}\subset \mathfrak{p}_2$. Hence there exists some $\sigma$ such that $\sigma(x)\in \mathfrak{p}_2$, in other word, $x\in \sigma^{-1}(\mathfrak{p}_2)$. So $\mathfrak{p}_1\subset \bigcup_{i}\sigma_i(\mathfrak{p}_2)$. By prime avoidance theorem, $\mathfrak{p}_1=\sigma(\mathfrak{p}_2)$ for some $\sigma\in G$.
Now, come back to our problem:
Let $I\subset \mathbb{Q}[x_1,\ldots,x_n,y]:=\mathbb{Q}[X,Y]$ be an ideal such that $\mathbb{Q}[X,Y]/I$ is of dimension zero. Suppose $L$ is a field finite over $\mathbb{Q}$ such that the maximal ideals $\mathfrak{m}_i,i=1,2,\ldots,l$ containing $I$ are $L$-coefficients. Write $\mathfrak{m}_i=(X-A_i,Y-b_i)$, and $P(Y)=(Y-b_1)(Y-b_2)\cdots(Y-b_l)$. We ask if $P(Y)\in \mathbb{Q}[Y]$?
The answer is Yes.
Let $F$ be a finite Galois extension of $\mathbb{Q}$, which contains $L$. Then it is clear to see that $(X-A_i,Y-b_i)$ are already maximal in $F[X,Y]$. So all maximal ideals in $F[X,Y]$ containing $I$ are exactly $\mathfrak{m}_iF[X,Y]$. Thus we may assume $L$ is a finite Galois extension of $\mathbb{Q}$.
Let $\sqrt{I}=\bigcap \mathfrak{n}_{i}$ in $\mathbb{Q}[X,Y]$. Let $S_i$ be the set of maximal ideals which lying over $\mathfrak{n}_i$ in $L[X,Y]$. Denoted $S$ the set of the maximal ideals $\mathfrak{m}_i$. Then $S$ is a disjoint union of $S_i$. And $G$ may act on $S$ having orbits $S_i$, $P(Y)=\prod_i P_i(Y)$ each $P_i(Y)\in \mathbb{Q}[Y]$.
Example Let us consider a concrete example. Consider $(x^2-2,y)$ in $\mathbb{Q}[x,y]$ is maximal, and $(x-\sqrt{2},y),(x+\sqrt{2},y)$ in $\mathbb{Q}[\sqrt{2}][x,y]$. Now our $P(y)=y^2$ which is not irreducible. However, this example suggests that $P(y)$ is a power of an irreducible polyonomial.
This is true!
lemma
Let $\mathfrak{n}$ be a maximal ideal of $\mathbb{Q}[X,y]$ and $L$ a finite Galois extension of $\mathbb{Q}$ such that the fiber of $\mathfrak{n}$ in $L[X,y]$ are all $L$-points. We construct $P(Y)$ as in the way above. Let $S$ be the set of maximal ideals $\mathfrak{m}_i=(X-a_i,Y-b_i)$ in $L[X,Y]$ which lying over $\mathfrak{n}$. Let $\mathfrak{n}\cap \mathbb{Q}[y]=(f(y))$ where $f(y)$ is a monic polynomial thus a monic irreducible polynomial. Then $P(Y)$ is a power of $f(y)$.
Proof
Observing that $G$ acts transitively on $S$, thus the group $H_i=\{\sigma\mid \sigma(b_i)=b_i\}$ are all conjugate. It follows that $P(y)=f(y)^{|H_1|}$. In particular, if all $b_i$ are different, then $P(y)=f(y)$.
Practice In practice, let $\alpha_1,\ldots,\alpha_s$ be algebraic numbers and $\beta=g(\alpha_1,\ldots,\alpha_s)$ for $g\in \mathbb{Q}[x_1,\ldots,x_s]$. We let $I=(f_1(x_1),\ldots,f_s(x_s),y-g(x_1,\ldots,x_s))$ where $f_i$ are minimal polynomials of $\alpha_i$ over $\mathbb{Q}$. If certain good conditions holds, for example, the extension $\mathbb{Q}[\alpha_1,\ldots,\alpha_i]/\mathbb{Q}[\alpha_1,\ldots,\alpha_{i-1}]$ has degree $deg(\alpha_i)$ for $i=1,\ldots,s$, then our $I$ is maximal. More, if we require $g(x_1,\ldots,x_s)$ are all distinct for substituting the roots of $f_i$. Then our $P(y)$ is irreducible polynomial of $\beta$ over $\mathbb{Q}$.
No Galois theory, no Nullstellensatz, but use ``The fundamental theorem of symmetric polynomials''.
Let $\alpha_1,\alpha_2,\ldots, \alpha_s$ be algebraic numbers. Let $\beta=g(\alpha_1,\ldots,\alpha_s)$ for some $g\in \mathbb{Q}[x]$. Let $f_i$ be the minimal polynomial of $\alpha_i$. Let $P(y)=\prod (y-g(\theta_1,\ldots,\theta_s))$ where $\theta_i$ runs through the roots of $f_i$. Then $P(y)$ is a polynomial of $\mathbb{Q}[y]$.
Proof
Let $\gamma_{i}$ be the roots of $f_1$. Then $\prod_i(y-g(\gamma_i,x_2,\ldots,x_s))$ is a symmetric polynoimal of $\gamma_i$ in $\mathbb{Q}[x_2,\ldots,x_s,y][\gamma_1,\ldots,\gamma_t]$ (View $\gamma_i$ being indeterminate). Since the element symmetric polynomials of $\gamma_1,\ldots,\gamma_t$ have vaule in $\mathbb{Q}$. Hence $\prod_i(y-g(\gamma_i,x_2,\ldots,x_s))$ is a polyonomial in $\mathbb{Q}[x_2,\ldots,x_s,y]$. Finally $P(y)\in \mathbb{Q}[y]$.
