Is $F(c)=F[c]$? I know that $F(c)$ and $F[x]/ \langle p(x) \rangle$ are isomorphic by a previous result proved in the book, what I would like to know is an explicit isomorphism between them so that I can get an idea of what these two fields look like. My reasoning goes as follows: 

First, we do not know what elements are in $F(c)$, that was the whole point of this construction, so our isomorphism must go from $F[x]/ \langle p(x) \rangle$ to $F(c)$.

$F[x]/ \langle p(x) \rangle= \{\langle p(x) \rangle+q(x) : q(x) \in F[x]\}$, so a natural idea to have is to set our function $\sigma (\langle p(x) \rangle+q(x)) = q(c)$. This function is a homomorphism and it is injective (if $q(c)=r(c)$ then $q(c)-r(c) \in \langle p(x) \rangle$, thus $\langle p(x) \rangle+q(x)=\langle p(x) \rangle+r(x)$). This gives me an idea of what $F(c)$ should look like (i.e. $\{ a(c) :  a(x) \in F[x]\}$), however, I get stuck when trying to prove surjectivity since I have no idea if there are elements of $F(c)$ that do not look like polynomials, and my attempts to prove it by contradiction fall short because all I know about $F(c)$ is that it is the smallest field containing both $F$ and $c$. 

I ran into this problem when trying to find an explicit description of $\mathbb Q (\sqrt 2)$, I found it online and it turned out to be $\{ a + b \sqrt 2 : a, b \in \mathbb Q\}$. This would be straightforward if I were able to prove that $\sigma $ is surjective, since every power of $\sqrt 2 $ is either $\sqrt 2 $ or $2$. 

Does this $\sigma $ work in general? And if it does, does that mean that every $F(c)=\{ a(c) :  a(x) \in F[x]\}$?
 A: Let me point out, to help you 'see' the rings better, that the element of F[c] are the linear combination in F of powers of c (same with F[x]) Notice that you could do the same for F(x) and F(c). (which makes you appreciate the notation :) )
Now for the explicit isomorphism that you asked for. (Sometimes it helps to go around in algebra rather than proving things directly)
(Of course I will assume F is field)
Consider the map $E_c: F[x] \to F[c]$ given by $E_c(f(x))=f(c)$
(exercise prove it is a surjective homomorphism)
Consider $ker(E_c)$. We know it is an ideal and since F is a field $F[x]$ is a PID.
So $ker(E_c)= (p(x))$ where $p(x) \ in F[x]$
Using the first isomorphism theorem we get $F[x]/(p(x)) \cong F[c]$
Now but also using (at least the proof of) the first isomorphism theorem you get that $E_c = f \circ \pi$ where $pi: F[x] \to F[x]/(p(x))$ projection map and we get $f(q(x) + (p(x))) = E_c(q(x))$
Thus the above shows that f is indeed the isomorphism you are looking for (now stated explicitly).
Now the fact that $F[x]/(p(x)) \cong F(c)$ follows from the fact that if c is algebraic over F, then we can write $ker(E_c)=(p(x))$ for a unique irreducible monic polynomial (exercise) and since we are in PID we get that $(p(x))$ is maximal (exercise: use prime ideal implies maximal in PID) so F[x]/(p(x)) is a field (also good exercise) and hence is isomorphic to its field of fractions.
