Can an integral domain have an element that has no square root but has a square root in the field of fractions? A lemma states: 

Let $R$ be a UFD and $F=\operatorname{Frac}(R)$. Let $d\in R$, then equation $a^2=d$ has a root in $R$ iff it has a root in $F$.

So I want to ask, is there a counterexample for this if $R$ is not a UFD?  I only know some non-UFD integral domains like ring of integers for some values.
 A: You already have a very good example, but since you mentioned ring of integers, and to put this in a more general context: 
A (full) ring of algebraic integers (a maximal order) can never work as a counter-example. The point is that those are (basically by definition) integrally closed. 
This means that every element of the fraction field of $R$ that is a root of a monic polynomial over $R$ is already in $R$. Since your equation corresponds to a particular type of monic polynomial having a root, the assertion holds for this one in particular. 
However, if you take subrings of rings of algebraic integers (non-maximal orders) you can get examples. For example, in $\mathbb{Z}[2\sqrt{2}]$ the equation $X^2 = 2$ has not solution. But $\sqrt{2}$  is of course in the quotient field. 
Let me end with an abstract argument for UFDs having the property you recalled: 
A UFD is integrally closed and thus it is also two-root closed (this is a somewhat common name for the property you give). 
Put differently, when looking for counter-examples you need to avoid integrally closed domains, since they all still have the property mentioned in your lemma. 
A: Let $k$ be a field and $R=k[x^2,x^3]$, the subring of $k[x]$ consisting of polynomials with no linear term.  Taking $d=x^2$, the equation $a^2=d$ has no root in $R$ since $x\not\in R$.  But $x=\frac{x^3}{x^2}$ is an element of the field of fractions $F$, so there does exist a root in $F$.
(To see directly that $R$ is not a UFD, note that $x^2$ and $x^3$ are both irreducible in $R$, but $(x^2)^3=(x^3)^2$, giving two distinct factorizations of $x^6$.)
