Is there a continuous function $f(x)$ such that the inverse function is $1/f(x)$? A student came to me with this question and I cracked my head for one hour but I couldn't unambiguously prove that it exists or it doesn't exist. Continuous and invertible function such that $f^{-1}(x) =1/f(x)$ on its domain of definition.
 A: Given

$$
f^{-1}(x) = \frac{1}{f(x)}. \tag 1
$$

Whence

$$
x = f^{-1}(f(x)) = \frac{1}{f(f(x))}. \tag 2
$$

Thus

$$
f(f(x)) = \frac{1}{x}.\tag 3
$$


Consider the function


$$
f(x) = \frac{1 + a x}{b + c x}. \tag 4
$$

Then

$$
f(f(x)) = \frac
    {\displaystyle 1 + a \frac{1 + a x}{b + c x}}
    {\displaystyle b + c \frac{1 + a x}{b + c x}} = \frac
    {[b + a] + [c + a^2] x}
    {[b^2 + c] + [bc + ac] x}. \tag 5
$$

So we get $f(f(x)) = x^{-1}$ for

$$
\left[
\begin{array}{rcl}
c + a^2 &=& 0\\
b^2 + c &=& 0\\
\displaystyle \frac{b+a}{bc+ac} &=& 1
\end{array} \tag 6
\right.
$$

Therefore $c=1$, $a=b=\pm \mathbf{i}$.
Whence

$$
\bbox[16px,border:2px solid #800000] {
   f(x) = \frac{1 \pm \mathbf{i} x}{x \pm \mathbf{i}}.} \tag 7
$$


Consider the function


$$
f(x) = x^\alpha. \tag 8
$$

Then

$$
f(f(x)) = \big( x^\alpha \big)^\alpha = x^{\alpha^2}. \tag 9
$$

So we get $f(f(x)) = x^{-1}$ for $\alpha = \pm \mathbf{i}$.
Whence

$$
\bbox[16px,border:2px solid #800000] {
   f(x) = x^{\displaystyle \pm \mathbf{i}}. } \tag {10}
$$

I have no idea if there are other possible function.
A: I think that it is difficult to fulfill. Since $f\circ f(x)=1/x$ we have that $f\circ f \circ f \circ f(x)=x$ therefore $f\circ f$ is an involution, therefore biyective. Therefore $f$ is biyective (?).
Note that we are asking for $f$ and $f^{-1}$ to be evaluated on the same $x$, therefore the domain of $f$ and of $f^{-1}$ has to be the same.
Hence its derivative (assuming is sufficiently nice function) does not change sign. So we have $(f\circ f)'(x)=f'(x)f'(f(x))>0$. Which is not the case since $f\circ f(x)=1/x$. Apparently I am saying that all doubly iterated monotonous functions are increasing. 
A: For this answer i will assume that the function is defined on an open interval (otherwise see the answer from @Elliot G ) and real.
Because $\frac1{f(x)}$ is the inverse function, we have $x = f^{-1}(f(x)) = \frac1{f(f(x))} $
and therefore $f(f(x)) = \frac 1 x$. 
Note that because $f$ is bijective, it has to be either monotone falling or increasing. if $f$ is increasing, so is $f(f(x))$, but $\frac 1 x$ is not increasing, so it is not possible.
if $f$ is decreasing, the composition $f \circ f$ is still increasing (because the composition of decreasing functions is increasing). So there is no way to do it on an open interval.
A: Consider the case with real variables.
Suppose $f$ is defined on an interval. Any continuous injection $(a,b)\to\Bbb R$ must be monotone and its square $f\circ f$ must be increasing. But the condition $f^{-1}(x)=f(x)^{-1}$ for all $x\in{\rm dom}(f)$ implies that $f(f(x))=x^{-1}$ holds on some interval, which is decreasing, a contradiction.

Consider the case with complex variables.
On the Riemann sphere $\widehat{\Bbb C}$ there is the Mobius transformation $\displaystyle f(z)=\frac{iz+1}{z+i}$.

In general one can define the question for topological groups. The functional condition on $f$ can be recast as a condition on its graph: $(x,y)\in{\cal G}\implies (y^{-1},x)\in{\cal G}$. I haven't been able to make much headway in this direction though. One issue is that I don't think there is any topological condition that characterizes those graphs in $X\times Y$ which are graphs of continuous functions $X\to Y$.
A: You might not like this, but consider $f:\{1\}\to\{1\}$ where $f(1)=1$. This satisfies the conditions since $f$ is technically continuous on the domain.
A: Briefly: Case i) $f$ is strictly increasing. Then $f^{-1}$ is strictly increasing, while $1/f$ is strictly decreasing. So there's no way the relation can hold. Case ii) $f$ is strictly decreasing. Same idea. Either case i) or case ii) must hold, so we're done.
