Solve the system $ x \lfloor y \rfloor = 7 $ and $ y \lfloor x \rfloor = 8 $. 
Solve the following system for $ x,y \in \mathbb{R} $: \begin{align} x \lfloor y \rfloor & = 7, \\ y \lfloor x \rfloor & = 8. \end{align}

It could be reducing to one variable, but it is not that simple.
Another idea is substitution, but it didnt lead me anywhere.
This was taken from a competition, and it’s tricky.
From what I noticed, systems with two or more variables involving "greatest integer" function are rarely seen, probably too difficult to analyze...

This was obtained with the help of Wolfram Alpha: (I swapped places of 7 and 8 by mistake, so it is not exactly the same system; anyway, the answer should not rely on software tools, of course)

 A: Note: Here we use the following characterization of the floor function 
\begin{align*}
\lfloor x\rfloor = p\qquad \text{ with } \qquad p \leq x < p+1, \quad p\in \mathbb Z
\end{align*}
We use this relationship to transform the system of equations into a system of inequalities which can be easily analysed.

Let $x,y \in \mathbb{R}$. The system of equations
  \begin{align*}
x \lfloor y \rfloor & = 7 \tag{1}\\
y \lfloor x \rfloor & = 8
\end{align*}
  is equivalent with
  \begin{align*}
x p & = 7 \qquad \text{ and }\qquad p\leq y < p+1, \quad p\in \mathbb{Z}\\
y q & = 8 \qquad \text{ and }\qquad q\leq x < q+1, \quad q\in \mathbb{Z}
\end{align*}
  We substitute $x=\frac{7}{p}$ and $y=\frac{8}{q}$ and get the following system  of inequalities with $p,q\in \mathbb{Z}$
  \begin{align*}
p\leq \frac{8}{q} < p+1\tag{2}\\
q\leq \frac{7}{p} < q+1\tag{3}
\end{align*}
Note that the two inequalities are equivalent with the two equations in (1).

We are now looking for integer solutions of these inequalities. First we see from (1) that $x\neq 0$ and $y\neq 0$. Then we deduce from (2) that either both $p$ and $q$ are greater zero or both are less than zero.

First step: $p,q>0$

Since $q > 0$ we conclude from (3) that $1\leq p \leq 7$ and we obtain
\begin{align*}
p=1 \qquad \rightarrow \qquad q\leq &\frac{7}{1}<q+1 \qquad \rightarrow \qquad q=7\\
p=2 \qquad \rightarrow \qquad q\leq &\frac{7}{2}<q+1 \qquad \rightarrow \qquad q=3\\
p=3 \qquad \rightarrow \qquad q\leq &\frac{7}{3}<q+1 \qquad \rightarrow \qquad q=2\\
p=4 \qquad \rightarrow \qquad q\leq &\frac{7}{4}<q+1 \qquad \rightarrow \qquad q=1\\
&\ldots\\
p=7 \qquad \rightarrow \qquad q\leq &\frac{7}{7}<q+1 \qquad \rightarrow \qquad q=1\\
\end{align*}

Checking these pairs $(p,q)$ in inequality (2) we see, that only $p=1$ and $p=2$ are valid.
\begin{align*}
(p,q)=(1,7) \qquad \rightarrow \qquad 1\leq \frac{8}{7} < 2\tag{4}\\
(p,q)=(2,3) \qquad \rightarrow \qquad 2\leq \frac{8}{3} < 3\\
\end{align*}
Since $x=\frac{7}{p}$ and $y=\frac{8}{q}$ we get from (4) the solutions with positive $x,y$
  \begin{align*}
A=\left\{\left(7,\frac{8}{7}\right),\left(\frac{7}{2},\frac{8}{3}\right)\right\}
\end{align*}

$$ $$

Second step: $p,q<0$
First we note, that if $p<-7$ it follows from (3) that $q=-1$. Putting $q=-1$ into (2) we obtain $p\leq -8<p+1$ and $p=-8$ follows.
We conclude that in the case $p<-7$ only $(p,q)=(-8,-1)$ is a valid solution.

Next we consider $-7 \leq p \leq -1$ similar to above
\begin{align*}
p=-1 \qquad \rightarrow \qquad q\leq &-\frac{7}{1}<q+1 \qquad \rightarrow \qquad q=-7\\
p=-2 \qquad \rightarrow \qquad q\leq &-\frac{7}{2}<q+1 \qquad \rightarrow \qquad q=-4\\
p=-3 \qquad \rightarrow \qquad q\leq &-\frac{7}{3}<q+1 \qquad \rightarrow \qquad q=-3\\
p=-4 \qquad \rightarrow \qquad q\leq &-\frac{7}{4}<q+1 \qquad \rightarrow \qquad q=-2\\
&\ldots\\
p=-7 \qquad \rightarrow \qquad q\leq &-\frac{7}{7}<q+1 \qquad \rightarrow \qquad q=-1\\
\end{align*}

Checking these seven pairs $(p,q)$ in (2) shows that the following three pairs are valid
  \begin{align*}
(p,q)=(-2,-4)\qquad\Rightarrow\qquad -2\leq- \frac{8}{4}<-1\\
(p,q)=(-3,-3)\qquad\Rightarrow\qquad -3\leq- \frac{8}{3}<-2\\
(p,q)=(-4,-2)\qquad\Rightarrow\qquad -4\leq- \frac{8}{2}<-3\\
\end{align*}
Together with $(-8,-1)$ we derive from $x=\frac{7}{p}$ and $y=\frac{8}{q}$ the following $(x,y)$ pairs with negative solutions
  \begin{align*}
B=\left\{\left(-\frac{7}{2},-2\right),\left(-\frac{7}{3},-\frac{8}{3}\right),\left(-\frac{7}{4},-4\right),\left(-\frac{7}{8},-8\right)\right\}
\end{align*}

Note that $A\cup B$ coincides with the solution of @AlexR.
A: Combining equations can work too. Eliminating $x$: $$y\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor=8\text{.}$$
In this equation, you are taking the floor of a rational number with $7$ as its numerator. If $y$ is large in absolute value ($8$ or above, or smaller than $-6$), then $\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor$ will either be $0$ or $-1$. So that leaves $y\in[-6,8)$. Of course $y$ cannot be in $[0,1)$ or the first original equation is false. 
For $y\in[-6,0)\cup[1,8)$, the possible values of $\left\lfloor y\right\rfloor$ are $\{-6,-5,\ldots,-1\}\cup\{1,2,\ldots,7\}$.
Running through these, the values of $\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor$ can only be $\{-2,-3,-4,-7,7,3,2,1\}$. So all together with the earlier noted $0$ and $-1$, $\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor$ can only take the values  $$\{-7,-4,-3,-2,-1,0,1,2,3,7\}\text{.}$$ 
Put these values back in to $$y=\frac{8}{\left\lfloor\frac{7}{\left\lfloor y\right\rfloor}\right\rfloor}$$ to determine that $$y\in\left\{-\frac87,-2,-\frac83,-4,-8,8,4,\frac83,\frac87\right\}$$
Since $x=\frac{7}{\lfloor y\rfloor}$, this corresponds to $$x\in\left\{-\frac72,-\frac72,-\frac73,-\frac74,-\frac78,\frac78,\frac74,\frac72,7\right\}$$ Out of these 9 possible solution pairs, checking each against the original two equations reveals that six are in fact solutions. The pairs that fail are $(-7/2,-8/7), (7/8,8)$, and $(7/4,4)$. So the solutions are $$\left\{\left(-\frac72,-2\right),\left(-\frac73,-\frac83\right),\left(-\frac74,-4\right),\left(-\frac78,-8\right),\left(\frac72,\frac83\right),\left(7,\frac87\right)\right\}$$
A: Let $x=m+\xi$, $y=n+\eta$ with $m$, $n\in{\mathbb Z}$, and $\xi$, $\eta\in[0,1[\ $. Then we have to solve $$(m+\xi)\>n=7,\qquad(n+\eta)\>m=8\ .\tag{1}$$
(i) If $m\geq1$, $n\geq1$ then $(1)$ implies
$$m=\left\lfloor{7\over n}\right\rfloor,\qquad n=\left\lfloor{8\over m}\right\rfloor\ .\tag{2}$$ The following table contains in the first row possible values of $n$, in the second row the resulting values of $m$ according to $(2_1)$, and in the third row the resulting values of $n$ according to $(2_2)$:
$$\matrix{
n&&1&2&3&4&5&6&7&\geq8 \cr
m:={\displaystyle\left\lfloor{7\over n}\right\rfloor} &&7&3&2&1&1&1&1&0\cr
{\displaystyle\left\lfloor{8\over m}\right\rfloor} &&1&2&4&8&8&8&8&-\cr}$$
A value $n$ in the first row is usable if the corresponding entry in the third row coincides with $n$. This leads to the usable pairs $(m,n)\in\bigl\{(7,1), (3,2)\bigr\}$, from which one computes the candidates
$$(x,y)\in\left\{\left(7,{8\over7}\right), \left({7\over2},{8\over3}\right)\right\}\ .$$
(ii) If $m':=-m\geq1$, $n':=-n\geq1$ then $(1)$ implies
$$(m'-\xi)\>n'=7,\qquad (n'-\eta)\>m'=8\ ,$$
or
$$n'=\left\lceil{8\over m'}\right\rceil,\qquad m'=\left\lceil{7\over n'}\right\rceil\ .\tag{3}$$
The following table contains in the first row possible values of $m'$, in the second row the resulting values of $n'$ according to $(3_1)$, and in the third row the resulting values of $m'$ according to $(3_2)$:
$$\matrix{ 
m' &&1&2&3&4&5&6&7&\geq8 \cr
n':={\displaystyle\left\lceil{8\over m'}\right\rceil} &&8&4&3&2&2&2&2&1\cr
{\displaystyle\left\lceil{7\over n'}\right\rceil} &&1&2&3&4&4&4&4&7\cr}$$
A value $m'$ in the first row is usable if the corresponding entry in the third row coincides with $m'$. This leads to the usable pairs $(m',n')\in\bigl\{(1,8), (2,4),(3,3),(4,2)\bigr\}$, from which one computes the candidates
$$(x,y)\in\left\{\left(-{7\over8},-8\right), \left(-{7\over4},-4\right),\left(-{7\over3},-{8\over3}\right),\left(-{7\over2},-2\right)\right\}\ .$$
It remains to check that all six candidate pairs $(x,y)$ that we have found indeed satisfy the original equations.
A: Reformulate: Let $a,b \in [0,1)$ and $m,n \in \mathbb Z$. Solve
$$(m+a)n = 7\\
(n+b)m = 8$$
Rearrangement gives
$$nm = 8 - bm = 7 - an$$
The product on the left is an integer so we already know that $bm, an\in \mathbb Z$ or in other words
$$a = \frac kn; \quad b = \frac lm$$
With $k,l\in\mathbb Z$. Substituting this back gives us
$$mn = 8-l = 7-k$$
Eliminating $l = k + 1$ so we get a solution by solving the integer equation
$$mn = 7-k; \quad k < \min(n, m-1)$$
Chose $k=1,m=3,n=2$ and get $l=2$ and finally
$$x = m + a = m + \frac kn = 3 + \frac12 = \frac 72\\
y = n + b = n + \frac lm = 2 + \frac 23 = \frac 83$$
Thanks to @Berrick Fillmore for the full set of solutions obtained from the last system:
$$\left\{ \left( 7,\frac{8}{7} \right), \left( \frac{7}{2},\frac{8}{3} \right), \left( - \frac{7}{8},-8 \right), \left( - \frac{7}{4},-4 \right), \left( - \frac{7}{3},- \frac{8}{3} \right), \left( - \frac{7}{2},-2 \right) \right\}$$
