Suppose a nonzero finitely generated module $M$ over $F$ is also finitely generated over $R$. Since $M$ is a vector space, it contains an $F$-submodule $L$ isomorphic to $F$. On the other hand there exists a surjective homomorphism $M\to L$ of $F$-modules, which is also a homomorphism of $R$-modules. Hence $L$ is also finitely generated over $R$.
Then the problem is reduced to showing whether $F$ is a finitely generated $R$-module.
Suppose $F$ is generated by $x_1/y,x_2/y,\dots,x_n/y$ over $R$, with $x_1,\dots,x_n,y\in R$; note that it is not restrictive to assume the same denominator. Then, for every $z\in F$ there are $r_1,\dots,r_n\in R$ such that
$$
z=\sum_{k=1}^n r_k\frac{x_k}{y}
$$
If $R\ne F$, we conclude that $y$ is not invertible in $R$. However
$$
\frac{1}{y^2}=\sum_{k=1}^n r_k\frac{x_k}{y}
$$
implies
$$
\frac{1}{y}=\sum_{k=1}^n r_kx_k\in R
$$
a contradiction.