My standard approach to a problem like this is to strive to keep things $~\color{red}{\text{linear}}~$ between $\epsilon$ and $\delta$. For this problem there are three artificial devices that facilitate this:
Given two points $(x_1,y_1)$ and $(x_2,y_2)$
let $D[(x_1,y_1),(x_2,y_2)]$ denote the (non-negative) distance between the two points.
That is,
$D[(x_1,y_1),(x_2,y_2)] = \sqrt{(x_1 - x_2)^2 + (y_1 - y_2)^2}.$
The first device is that
$0 < D[(x_1,y_1),(x_2,y_2)] < \delta \implies $
$0 \leq |x_1 - x_2| < \delta ~~\text{and}~~ 0 \leq |y_1 - y_2| < \delta.$
For the second device, suppose that $A,B,C,D$ are all positive numbers.
Further suppose that you have two variables, that I shall call $u$ and $v$.
Further suppose that you know that
$A < u < B~~$ and $~~C < v < D.$
Then you can immediately conclude that
$\displaystyle \frac{A}{D} < \frac{u}{v} < \frac{B}{C}.$
The idea is that you minimize $~\dfrac{u}{v}~$ by minimizing its numerator and maximizing its denominator.
You maximize $~\dfrac{u}{v}~$ by the reverse process.
Given $\epsilon > 0$, when you are constructing the relationship between $\delta$ and $\epsilon$, add an additional maximum-value constraint on $\delta$.
For example, in this problem, I will impose the constraint that $\delta$ must be $\leq \dfrac{1}{10}.$
This implies that (for example) $\delta^2 \leq \dfrac{\delta}{10}.$
This helps to keep things linear, between $\epsilon$ and $\delta.$
The challenge is to establish a relationship between $\epsilon$ and $\delta$ such that
$$0 < D\left[ ~(x,y),\left(~\sqrt{1/2},\sqrt{1/2}~\right) ~\right] < \delta \implies
~\left|\frac{x^2 + y^2 - 1}{xy} - 0 ~\right| < \epsilon.$$
Note
There are much easier demonstrations of the posted assertion, that in no way involve $\epsilon,\delta$. For example, it is clear that in the posted question, both the numerator and denominator are continuous functions, where the limit as $(x,y)$ approaches the pertinent point of the numerator is $(0)$ and the corresponding limit of the denominator is not zero.
Depending on what theory that you were taught, this makes it game over. However, the OP (i.e. original poster) specifically mentioned that $\epsilon$ and $\delta$ should be used. So, I will attack the problem from that viewpoint.
As discussed, since the distance between $(x,y)$ and $~\displaystyle \left(~\sqrt{1/2},\sqrt{1/2}~\right) ~$ is between $(0)$ and $(\delta),$ exclusive, you have that
This implies that $x,y$ are both bounded by:
This implies that
$$~\frac{1}{2} - 2\delta\sqrt{1/2} + \delta^2 < x^2 < \frac{1}{2} + 2\delta\sqrt{1/2} + \delta^2.$$
Since $\delta \leq \dfrac{1}{10}$,
this implies that
$$~\frac{1}{2} - 2\delta\sqrt{1/2} < x^2 < \frac{1}{2} + \delta\left[2\sqrt{1/2} + \frac{1}{10}\right].$$
Further, since $~\displaystyle 2\sqrt{1/2} < 1.5~$
I can immediately conclude that
$$\frac{1}{2} - 2\delta < x^2 < \frac{1}{2} + 2\delta. \tag1 $$
By parallel analysis applied to the variable $(y)$, I can similarly conclude that
$$\frac{1}{2} - 2\delta < y^2 < \frac{1}{2} + 2\delta. \tag2 $$
This means that I know that the numerator,
$(x^2 + y^2 - 1)$ is bounded by
$$-4\delta < (x^2 + y^2 - 1) < 4\delta. \tag3 $$
Now, it is time to establish bounds on the denominator, $(xy)$.
Note that in the (small) neighborhood of radius $\delta$ around $~\displaystyle \left(\sqrt{1/2},\sqrt{1/2}\right),~$ that both of the coordinates of $(x,y)$ are positive.
Further, using (1) and (2) above, I have that
$$\left[\frac{1}{2} - 2\delta\right]^2 < (xy)^2 < \left[\frac{1}{2} + 2\delta\right]^2.$$
This implies [since $x,y$ are each known to be positive], that
$$\frac{1}{2} - 2\delta < (xy) < \frac{1}{2} + 2\delta. \tag4 $$
It only remains to
put (3) and (4) above together
use this to determine upper and lower bounds on the fraction, in terms of $\delta$
use these bounds to construct the appropriate relationship between $\epsilon$ and $\delta.$
Since $0 < \delta \leq \dfrac{1}{10}$, I can immediately conclude that
$$0.3 \leq \frac{1}{2} - 2\delta < (xy) < \frac{1}{2} + 2\delta \leq 0.7. \tag5 $$
Since $\dfrac{1}{4} < 0.3$, I can therefore, immediately conclude that
$$\dfrac{1}{4} < (xy) < 1.$$
Now, I can use this in conjunction with (3) above.
I can now conclude that
$$\frac{-4\delta}{(1/4)} < \frac{x^2 + y^2 - 1}{xy} < \frac{4\delta}{(1/4)} \implies $$
$$-16\delta < \frac{x^2 + y^2 - 1}{xy} < 16\delta.$$
Therefore, I can conclude that the following relationship between $\delta$ and $\epsilon$ will be satisfactory:
$$\delta = \min\left[ ~\frac{\epsilon}{16}, ~\frac{1}{10} ~\right].$$