How should I solve problems like this? This is from a math competition for elementary school kids.
x, y, and z are positive integer and they satisfy

The question ask for the value of -x-y+z
I do not know how to solve this
I can simplify this to
$$\frac{y}{xy+1}+z=\frac{5xy}{7}$$
Then what?
I can go further multiply both sides by $7(xy+1)$
I will get let's see
$$7y+7xyz+7z=5xy(xy+1)$$
Then I do not know what to do
 A: Since $xy + 1$ and $y$ are relatively prime, we must have $xy + 1 = 7$. So $xy = 6$, right hand side is $30/7$ and $z = 4$.
A: I think OP wants this answered in a way elementary school kids would understand.

If you don't know how to solve it, try making the expression nicer and looking for "clues". I'll be doing that here: I'll assume I do not know number theory, nor how to solve this in any rigorous way. I'll assume we do know that having a factor on one side of the equation implies having it on the other as well.
Since $-x-y+z$ a combination is wanted, we need to either re-write the original problem as that, or find at least one solution for $x,y,z$. 


When presented with such a problem, I would always first simplify individual terms, as you did:
$$
\frac{y}{xy+1}+z=\frac{5xy}{7}
$$
Then I would get rid of fractions. That is, multiply by denominators:
$$
7y+7z(xy+1)=5xy(xy+1)
$$
Finally I would want to factorize in a nice way. Notice "($xy+1$)" on both sides. Lets try to factor that:
$$
7y=5xy(xy+1)-7z(xy+1) \\
7y=(xy+1)(5xy-7z)
$$
If you've factorized this in a nice way, we should start getting clues.
Notice LHS has a factor $7$. This means RHS must have too. That is:
For $(xy+1)(5xy-7z)$ to have a factor of $7$, either $(xy+1)$ or $(5xy-7z)$ must have a factor of $7$. 
Notice the second case  is a combination of a number $[5xy]$ and a multiple of $7$ that is $[7z]$. This means that number $[5xy]$ must be a multiple of $7$ so that their combination $([5xy]-[7z])$ can be a multiple of $7$, and by that have a factor of $7$.
We now have either $(xy+1)$ or $(5xy)$ must have a factor of $7$.
We can now examine both cases. 

First case:
Lets assume the " factor (multiple) of $7$" is the smallest such and keep trying to find smallest solutions.
That is, this gives us $xy+1=7\implies xy=6$. Here, we have $\{x,y\}$ can be $\{1,6\}$ or $\{2,3\}$ to give $6$ when multiplied.
We have: (Use the fact that $xy=6$)
$$\begin{array}{}
7y&=(xy+1)(5xy-7z) \\
7y&=(6+1)(5\cdot 6 -7z) \\
7y&=7(30-7z) \\
y&=30-7z \\
7z&=30-y
\end{array}$$
Notice $y$ is either $1,2,3,6$ and only $2$ works. In that case, we have $\{2,3\}$ so $x=3$.
We now have $z$ as well:
$$
7z=2\cdot14\implies z=4
$$
And we see that $-x-y+z=-3-2+4=-1$ the answer is $-1$.
Notice we did not need to try the second case as we've already found a solution here. 
If you have started with the second case first, you might have gotten stuck, since that case does not have solutions. In that scenario, if you get "stuck", try the other case - the first case, which in this scenario has a solution.
A: $x y + 1 = 7 → x y = 6$
Matching fractional parts:  $y ≡ 5 x y \text{ (mod 7)}$
$ 5x ≡ 1 ≡ 15 \text{ (mod 7)} → x=3 → y=2  → z=4$
$-x-y+z = -3-2+4 = -1$
