If b is algebraic over F(a), and a is algebraic over F, then is b algebraic over F? 
If $b$ is algebraic over the field $F(a)$ then is it algebraic over the field F?

I would like to find a proof if it is true or a counter example if it is not.
The only thing I could think of was trying to show that if b is a zero of a polynomial $p(x)$ in $F(a)$ then there would be a way to turn it into a polynomial in $F$ which $b$ is also a zero of.
 A: Counterexample: We can extend $\mathbb{Q}$ by $\pi$. In this case $\pi$ is algebraic over $\mathbb{Q}[\pi]$ but not over $\mathbb{Q}$. 
A: Let $K = F(a)$.  Since $K$ is a field containing $F$, $K$ is a vector space over $F$.  Assume $a$ is algebraic over $F$.  In this case, we have:
Lemma: If $F$ is a field, and $c$ is algebraic over $F$, then the field $F(c)$ is a finite dimensional vector space over $F$.
I hope you've already been introduced to this lemma, if not, it's a good exercise which makes use of the division algorithm.  Anyway, $K$ is a finite dimensional vector space over $F$.
Let $L$ be the field $K(b)$. Since $b$ is algebraic over $K$ by hypothesis, you can apply the lemma again, and so $L$ is a finite dimensional vector space over $K$.  
Lemma 2: If $F_1 \subseteq F_2 \subseteq F_3$ are fields, then $\textrm{Dim }_{F_1} F_3 = (\textrm{Dim }_{F_1} F_2)\cdot (\textrm{Dim }_{F_1} F_3)$.
This isn't difficult to prove if you haven't seen it before.  You can just show that if $\{v_i\}$ is a basis for $F_2$ over $F_1$, and $\{w_j\}$ is a basis for $F_3$ over $F_2$, then the set of products $\{v_iw_j\}$ forms a basis for $F_3$ over $F_1$.  Apply linear independence in two steps.  Anyway, by this lemma, $L$ is a finite dimensional vector space over $F$ with dimension say, $m$.
Now $L$ is a field containing the element $b$.  The $m+1$ elements $1, b, b^2, ... , b^m \in L$ are linearly dependent over $F$, so there must exist scalars $c_0, c_1, ... , c_m \in F$, not all zero, such that $$c_0 + c_1b + \cdots + c_mb^m = 0$$ But now $b$ is a root of the nonzero polynomial $f(X) = c_0 + c_1X + \cdots + c_mX^m$, which lies in $F[X]$.  Let me know if you have any questions. 
