A self inverse function $f(x)=\frac{ax+1}{x-b}$ 
If  $$f(x)=\frac{ax+1}{x-b} \forall x \in\mathbb{R}-b,ab\neq1,a\neq1$$ is a self inverse function such that $$\frac{f(4)}{4}=\frac{f(12)}{12}={f\left(\frac{1+b}{1-a}\right)}$$The question is to find out $a$ and $b$

For a self inverse function $f(f(x))=x$.So I tried to put $f(x)$ in place of $x$ and solve the resulting equation but it didnot helped me .Is there a more logical way to solve this problem in limited time?Any ideas?Thanks.
 A: Write this as

$$\begin{pmatrix}
a && 1 \\
1 && -b
\end{pmatrix}\begin{pmatrix}x \\ 1\end{pmatrix}$$

Then this is self inverse iff
$$\begin{pmatrix}
a && 1 \\
1 && -b
\end{pmatrix}^2= \lambda I_2$$
i.e.
$$\begin{pmatrix}
a^2+1 && a-b \\
a-b && 1+b^2
\end{pmatrix}= \lambda I_2$$
So $a=b$.
The transformation is projective so all we need now is to check $a,b$ so that your conditions follow.
$$12f(4)=4f(12)\iff 3(4a+1)/(4-a)=(12a+1)/(12-a)$$
$$\iff 3(4a+1)(12-a)=(12a+1)(4-a)$$
$$\iff 143a+36=47a+4$$
$$\iff 96a=-32\iff a=-1/3.$$
Finally
$$\displaystyle f\left({1+b\over1-a}\right)=f(1/2)=1$$
$$\ne -1/52=f(4)/4.$$
So no such transformation exists.
A: Suppose $b\neq 0$. Then $f(0)=-1/b$ and if $f$ is self inverse
$$0=f(-1/b)=\frac{a/b-1}{b+1/b}=\frac{a-b}{b^2+1}.$$
It follows that $a=b$. Then
$$f(f(x))=\frac{a\frac{ax+1}{x-a}+1}{\frac{ax+1}{x-a}-a}=\frac{a^2x+a+(x-a)}{ax+1-a(x-a)}=\frac{a^2x+x}{1+a^2}=x$$
and $f$ is self-inverse. It suffices now to check the remaining conditions. We have that
$$f\left(\frac{1+a}{1-a}\right)=\frac{a\frac{1+a}{1-a}+1}{\frac{1+a}{1-a}-a}=1$$
so it must be that
$$\frac{\frac{4a+1}{4-a}}{4}=\frac{\frac{12a+1}{12-a}}{12}=1$$
But both equations cannot be satisfied simultaneously.

Now, if $b=0$, then $f(x)=a+\frac{1}{x}$ and hence
$$f(f(x))=a+\frac{1}{a+\frac{1}{x}}=a+\frac{x}{ax+1}=\frac{a^2x+x+a}{ax+1}$$
And if $f$ is self inverse
$$a^2x+x+a=ax^2+x\implies ax^2-a^2x-a=0$$
And unless $a=0$, this can be true for at most two distinct values of $x$, so the only option for $f$ to be self inverse is $f(x)=1/x$. Howevr, the condition on $f(4)$ and $f(12)$ is once again not satisfied.
Perhaps it was wrongly transcribed? The following relations do hold in this case:
$$4\cdot f(4)=12\cdot f(12)=f\left(\frac{1+b}{1-a}\right)$$
A: 
Is there a more logical way to solve this problem in limited time?

Hint: if you take the premise to mean that there do in fact exist $a,b$ such as $f=f^{-1}$ then, for a shortcut, you can simply plug in $x=0$ and determine that $f\big(f(0)\big)=f(\frac{-1}{b})=0 \implies a=b\,$. Then use the second condition to determine $a\,$ or, as is the case, prove that no solutions exist.
