I can't derive the integrating factor of this first order ODE from the Dover textbook I'm a junior mechanical engineering student. 
I can't derive the integrating factor of this first order ODE
$(x^ 2  - y^2  - y) dx - (x^ 2  - y^2  - x) dy  =  O$
The textbook provides 5 integrating factor formulas. I tried to use the one where u =y/x, k(u) = x2[(d/dx)Q(x,y) - (d/dy)P(x,y)] / [xP(x,y) + yQ(x,y)]  but I  still had an x left in the denominator.
I'm stuck hard on this one.
 A: Solve $(x^2-y^2-y)\,dx+(y^2-x^2+x)\,dy=0$
Let $z=x-y$. Then $x+y=2x-z$ so $x^2-y^2=2xz-z^2.$ 
Substituting gives
\begin{equation}
(2xz-z^2-x+z)\,dx+(z^2-2xz+x)(dx-dz)=0
\end{equation}
which simplifies to
\begin{equation}
z\,dx+(2xz-z^2-x)\,dz=0
\end{equation}
So
\begin{eqnarray}
\dfrac{\partial M}{\partial{z}}&=1\\
\dfrac{\partial N}{\partial{x}}&=2z-1
\end{eqnarray}
Therefore
\begin{equation}
\dfrac{1}{M}\cdot\left(\dfrac{\partial N}{\partial{x}}-\dfrac{\partial M}{\partial{z}}\right)=2-\dfrac{2}{z}
\end{equation}
So $\mu=z^{-2}e^{2z}$ is an integrating factor and
\begin{equation}
z^{-1}e^{2z}\,dx+(2xz^{-1}-1-xz^{-2})e^{2z}\,dz=0
\end{equation}
is exact since
\begin{equation}
\dfrac{\partial}{\partial z}\left(z^{-1}e^{2z}\right)=(-z^{-2}+2z^{-1})e^{2z}=\dfrac{\partial}{\partial x}\left(2xz^{-1}-1-xz^{-2}\right)e^{2z}
\end{equation}
Integrating the respective terms with respect to $x$ and $z$ yields
\begin{equation}
\phi(x,z)=xz^{-1}e^{2z}+C(z)
\end{equation}
and
\begin{equation}
\phi(x,z)=xz^{-1}e^{2z}-\frac{1}{2}e^{2z}+C(x)
\end{equation}
So the general solution in terms of $x$ and $z$ is
\begin{equation}
\left(\dfrac{x}{z}-\frac{1}{2}\right)e^{2z}=C
\end{equation}
Therefore the solution in terms of $x$ and $y$ is
\begin{equation}
\left(\dfrac{x}{x-y}-\frac{1}{2}\right)e^{2(x-y)}=C
\end{equation}
which can be simplified to
\begin{equation}
\left(\dfrac{x+y}{x-y}\right)e^{2(x-y)}=c   
\end{equation}
A: Alright this problem is interesting. Here's the spoilers: The integrating factor your seek is the function
$$\mu(x,y)=\frac{e^{2(x-y)}}{(y-x)^2}.$$
I tested with Mathematica that indeed this integrating factor will make your ODE exact. How to obtain it? Here's the procedure I used, which does not use any formula (since I don't know what book you refer to and I don't know any formulas), but only makes use of what we know we're looking for. Below is the main ideas:
So we want a function $\mu(x,y)$ which, when multiplied to your ODE, makes it exact. That means our $\mu$ must satisfy
$$
\mu_yM+\mu M_y=\mu_x N+\mu N_x.
$$
Immediately there seems to be some symmetry that we can exploit, so assume that $\mu_y=-\mu_x$. Under this assumption, the above equation reduces to
$$
(y-x)\mu_x=2(1+y-x)\mu,
$$
which is a separable first order ODE on $x$ (edit to clarify: think of $y$ as a constant, the unknown is $\mu$). Upon solving it and letting the "arbitrary" function of $y$ that shows up be $2y$ (which is clear is needed because we assumed $\mu_y=-\mu_x$), you get our $\mu$.
From here on it shouldn't be conceptually difficult, although the math itself may be painful. Best of luck!
A: $$(x^ 2  - y^2  - y) dx - (x^ 2  - y^2  - x) dy  =  0 \qquad\qquad [1]$$
Let $\mu(x,y)$ denotes the integrating factor, so that the equation be the total differential of a function $F(x,y)$
$$\mu(x^ 2  - y^2  - y) dx - \mu(x^ 2  - y^2  - x) dy  =  dF \qquad\qquad [2]$$
$$\begin{cases}
\frac{\partial F}{\partial x}=\mu(x^ 2  - y^2  - y)\\
\frac{\partial F}{\partial y}= -\mu(x^ 2  - y^2  - x)
\end{cases}$$
with condition to be a total differential : 
$$\frac{\partial^2 F}{\partial x\partial y}=\frac{\partial }{\partial y}\left(\mu(x^ 2  - y^2  - y)\right)=\frac{\partial }{\partial x}\left( -\mu(x^ 2  - y^2  - x)\right)$$
This leads to a first order linear PDE :
$$(x^2-y^2-x)\frac{\partial \mu}{\partial x}+(x^2-y^2-y)\frac{\partial \mu}{\partial y} = 2(y-x+1)\mu$$
Method of characteristics : The set of characteristic differential equations is :
$$\frac{dx}{x^2-y^2-x}=\frac{dy}{x^2-y^2-y}=\frac{d\mu}{2(y-x+1)\mu}$$
Thanks to the well-known general relationship $\frac{a}{b}=\frac{c}{d}=\frac{a-c}{b-d}$ : 
$\frac{dx}{x^2-y^2-x}=\frac{dy}{x^2-y^2-y}= \frac{dx-dy}{(x^2-y^2-x)-(x^2-y^2-y)}=\frac{dx-dy}{-x+y}=-\frac{d(x-y)}{x-y}$
The equation of a characteristic curve comes from : $-\frac{d(x-y)}{x-y}=\frac{d\mu}{2(y-x+1)\mu}$
With $Y=x-y \quad\to\quad -\frac{dY}{Y}=\frac{d\mu}{2(-Y+1)\mu} \quad\to\quad -2(-1+\frac{1}{Y})dY=\frac{d\mu}{\mu}$
It is easy to solve this linear ODE. $\quad \ln(\mu)=2Y-2\ln(Y)+$constant. 
Any one of the solutions is a convenient integrating factor :
$$\mu=e^{2Y}Y^{-2}=\frac{e^{2(x-y)}}{(x-y)^2}$$
With this integrating factor it is easy to integrate the above equation $[2]$ with $dF=0$ : 
$$\frac{y+x}{y-x}e^{2(x-y)}=C$$
This is the solution of equation $[1]$ expressed on implicit form. As far as I know, there is no closed form to express explicitly the function $y(x)$.
A: To find an $f$ so that
$$
\left(x^2-y^2-y\right)f\,\mathrm{d}x-\left(x^2-y^2-x\right)f\,\mathrm{d}y=0\tag{1}
$$
is exact, we need to have
$$
\underbrace{-(2y+1)f+\left(x^2-y^2-y\right)\partial_yf}_{\partial_y\left(x^2-y^2-y\right)f}=\underbrace{-(2x-1)f-\left(x^2-y^2-x\right)\partial_xf}_{-\partial_x\left(x^2-y^2-x\right)f}\tag{2}
$$
which becomes
$$
\left(x^2-y^2-y\right)\partial_y\log(f)+\left(x^2-y^2-x\right)\partial_x\log(f)=2y+2-2x\tag{3}
$$
Note that
$$
\left(x^2-y^2-y\right)\partial_y(x-y)+\left(x^2-y^2-x\right)\partial_x(x-y)=y-x\tag{4}
$$
and
$$
\left(x^2-y^2-y\right)\partial_y(\log(x-y))+\left(x^2-y^2-x\right)\partial_x(\log(x-y))=-1\tag{5}
$$
Therefore, subtracting $2$ times $(5)$ from $2$ times $(4)$, we find that if we set
$$
\log(f)=2(x-y-\log(x-y))\tag{6}
$$ we have $(3)$. Thus,
$$
\bbox[5px,border:2px solid #C0A000]{f(x,y)=\left(\frac{e^{x-y}}{x-y}\right)^2}\tag{7}
$$
is an appropriate integrating factor for $(1)$.

Judging from the fact that the integrating factor is a function of $x-y$, it might be interesting to look at the change of variables $u=x+y$ and $v=x-y$ (rotation by $45$ degrees). Then, $(1)$ becomes
$$
\begin{align}
0
&=\left(x^2-y^2-y\right)\,\mathrm{d}x-\left(x^2-y^2-x\right)\,\mathrm{d}y\\
&=\left(uv-\frac{u-v}2\right)\,\mathrm{d}\frac{u+v}2-\left(uv-\frac{u+v}2\right)\,\mathrm{d}\frac{u-v}2\\
&=\frac{v}2\,\mathrm{d}u+\left(uv-\frac{u}2\right)\mathrm{d}v\tag{8}
\end{align}
$$
which is exact when multiplied by an $f$ so that
$$
\frac{v}2\partial_v\log(f)+\left(\frac{u}2-uv\right)\partial_u\log(f)=v-1\tag{9}
$$
If we make $f$ a function of $v$ only, $(9)$ becomes
$$
\frac{\mathrm{d}}{\mathrm{d}v}\log(f)=2-\frac2v\tag{10}
$$
and $(10)$ has a solution
$$
f(v)=\left(\frac{e^v}{v}\right)^2\tag{11}
$$
which agrees with $(7)$.
