Find the parameters $a$ and $b$ of the function $f(x, y)=\dfrac{ax+by}{1+xy}$ given that the function is a binary operation on $(-1, 1)$. I am given the function 
$$f(x, y) = \dfrac{ax+by}{1+xy}$$
With $a, b \in \mathbb{R}$. Also it is known that the function $f(x, y)$ is a binary operation on the interval $(-1, 1)$, so $\forall$ $x, y \in (-1, 1)$ $\Rightarrow f(x, y) \in (-1, 1)$. I have to find the values of $a$ and $b$ and I am given the following options:
A. $a = b = 2$
B. $a + b \in (-1, 1)$
C. $a \in (-1, 1)$ and $b \in (-1, 1)$
D. $a = b \in [-1, 1]$
E. $a + b = 1$
Now I checked the function $f(x, y) = \dfrac{x+y}{1+xy}$ (so with $a=b=1$) and I found that this function is indeed a binary operation on the interval $(-1, 1)$. The only possible answer that contains $1$ is answer D so I know that this is the right answer. But I am curious as to how can I find that this is the answer by calculations, rather than by guessing.
 A: Suppose $a\ne b$, and put $x=1-\epsilon,y=-1+\epsilon$ (the idea here is to make $1-xy$ 
very small, so that the fraction will be large). We get
$$\frac{ax+by}{1+xy}=\frac{(a-b)(1-\epsilon)}{\epsilon(2-\epsilon)}$$
And if $a\ne b$, we can make this as large as we like in absolute value by choosing $\epsilon$ small enough. In particular, we can make it $> 1$ in absolute value.
Therefore $a=b$, and we can write the fraction as
$$\frac{a(x+y)}{1+xy}$$
Now we need to show that $|a|\le 1$.
If $|a|>1$, then putting $x=y=1-\epsilon$ gives
$$\frac{a(2-2\epsilon)}{2-2\epsilon+\epsilon^2}$$
which can be made $>1$ in absolute value by choosing $\epsilon$ small enough.
And if $|a|\le 1$, then your own work shows that the fraction is $< 1$.
A: Notice that as you approach either the point $(1,-1)$ or $(-1,1)$ the denominator goes to $0$. In order for this function to have any hope of having a bounded range is for the limits
$$\lim_{(x,y)\to(1,-1)} ax+by \hspace{20 pt} \lim_{(x,y)\to(-1,1)} ax+by$$
to both go to $0$. The only possible solution is if $a=b$. Then the choice of values of $a$ and $b$ mark what the range of of the whole function should be. Realistically, since this was multiple choice, your work would stop here because this simple observation only has one possible answer among the choices.
If you wanted to continue, simple optimization shows that the critical points of this function occur when $x^2=y^2=1$ so we get that
$$\lim_{(x,y)\to(\pm 1,\pm 1)} \frac{a(x+y)}{1+xy} = \pm a$$
$$\lim_{(x,y)\to(\pm 1,\mp 1)} \frac{a(x+y)}{1+xy} = 0$$
Having the image be $(-1,1)$ restricts $a$ to be from the same interval, but including the endpoints since you aren't actually allowed to plug in the maximum, so having $a=1$ is okay for $x,y\in(-1,1)$.
