Polynomial coefficients in subfield if all evaluations in subfield? I'm reading through a proof on central simple algebras and the following seems to be implicitly assumed:

Let $L/K$ be a field extension, $F \in L[X_1, \ldots, X_n]$ a
  polynomial such that $F(k_1, \ldots, k_n) \in K$ for all $(k_1, \ldots, k_n) \in K^n$. Then the coefficients of $F$ actually lie in $K$.

Is this true in the form stated above, or under additional assumptions? In the proof I am reading, $F$ is homogeneous and $L/K$ is a finite extension.
 A: Take $K$ to be the field with $p$ elements, $p$ a prime, $L$ to be the field with $p^{2}$ elements, and let $\alpha \in L \setminus K$.
The polynomial
$$F = \alpha (X^{p} - X)$$
is identically zero for all evaluations of $X$ in $F$. But clearly $F \notin K[X]$.
But then perhaps you are working in characteristic zero? Or, as suggested by user26857, $K$ is infinite?
A: A proof of the following statement:

Let $K$ be a field, $L/K$ a field extension. Let $F \in L[X_1, \ldots, X_n]$ be a polynomial such that $F(k_1, \ldots, k_n) \in K$ for all $(k_1, \ldots, k_n) \in K^n$ and $\deg(F) < \lvert K \rvert$. Then the coefficients of $F$ lie in $K$.

By induction on $n$. If $n=0$ then $F$ is a constant polynomial and there is nothing to show. We consider the case $n=1$. Let 
$$
F(X) = \sum_{i=0}^d g_iX^i
$$
where $d = \deg(F) < \lvert K \rvert$ and $g_0, \ldots, g_d \in L$. We can use Lagrange interpolation (user26857 mentioned that this works when the characteristic is 0, but I only need that $d < \lvert K \rvert$). Indeed, just pick different elements $x_0, \ldots, x_d \in K$ and define the polynomial
$$
l_j(X) = \prod_{m=0, m \neq j}^d \frac{X - x_m}{x_j - x_m} \in K[X]
$$
for $j \in \lbrace 0, \ldots, d\rbrace$. $l_j$ is a degree $d$ polynomial and one has $l_j(x_i) = \delta_{i, j}$. Because the degree $d$ polynomial
$$
F(X) - \sum_{j=0}^d F(x_j) l_j(X)
$$
has at least $d+1$ distinct roots (namely $x_0, \ldots, x_d$) the polynomial must be identically zero. Hence we find that
$$
F(X) = \sum_{j=0}^d F(x_j) l_j(X) \in K[X]
$$
which concludes the proof in the case $n = 1$.
Suppose now $n > 1$, write
$$
F(X_1, \ldots, X_n) = \sum_{i=0}^d F_i(X_1, \ldots, X_{n-1})X_n^i
$$
with $d = \deg(F) < \lvert K \rvert$ and $F_i \in L[X_1, \ldots, X_{n-1}]$ with $\deg(F_i) \leq d$. This time we argue via contraposition. If not all coefficients of $F$ lie in $K$, then pick $i_0 \in \lbrace 0, 1, \ldots, d \rbrace$ such that the coefficients of $F_{i_0}$ do not all lie in $K$. By induction hypothesis we can pick $(k_1, \ldots, k_{n-1}) \in K^{n-1}$ such that $F_{i_0}(k_1, \ldots, k_{n-1}) \not\in K$. Then not all coefficients of the polynomial in one variable
$$
G(X) = \sum_{i=0}^d g_iX^i
$$
where $g_i = F_i(k_1, \ldots, k_{n-1})$ lie in $K$. By the case $n=1$ this implies that $G(k_n) \not\in K$ for some $k_n \in K$. Hence $F(k_1, \ldots, k_n) \not\in K$. $\square$
