How to solve equations in the form $af^2(x)+bf(x)+cx=0$?

I have not seen any books about solving the equation of the following form: $af^2(x)+bf(x)+cx=0$ where $a$, $b$, $c$ are constants and $f^2(x)=f(f(x))$. We are going to find an expression of the function $f$. If I substitute $f(x)=kx$ in, I can get a solution, but I am interested in how to get ALL the solutions.

Please assume continuity if necessary.

Does any one know how to solve such functional equation?

• I don't think you can expect nice formulae for all solutions. In the case $a=1$, $b=0$, and $c=-1$, the solutions $f$ are involutions, which can be described by partitioning $\mathbb{R}$ into singletons and subsets of size $2$. If $\{t\}$ is a singleton in such a partition, we set $f(t):=t$; if $\{p,q\}$ is a $2$-subset in this partition, we set $f(p):=q$ and $f(q):=p$. – Batominovski Sep 20 '18 at 5:31
• In the case $a=b=c=1$, you partition $\mathbb{R}$ into sets of the form $\{p,q,-p-q\}$ (where either $p$, $q$, and $-p-q$ are pairwise distinct, or both $p$ and $q$ are $0$). Then, for a subset $\{p,q,-p-q\}$ in this parition, we define $f(p):=q$, $f(q):=-p-q$, and $f(-p-q):=p$. However, if you demand that the solutions be continuous, then you may be able to show that all solutions are linear (I am guessing here, so don't blindly believe that this last sentence must hold). – Batominovski Sep 20 '18 at 5:39
• What is wrong with solving the quadratic equation $ay^2+by+cx=0$? The other constraint is more demanding though. – Arash Sep 20 '18 at 7:43
• @Arash: That's what I thought, too. The problem is that, as the OP has mentioned, $f^2(x)=f(f(x)),$ not $f^2(x)=(f(x))(f(x)).$ The quadratic equation doesn't apply to that sort of thing, unfortunately. – Adrian Keister Sep 24 '18 at 17:29

The particular case of your equation

$$f^2(x)=x$$ is known as Babbage's functional equation.

If you have found a solution (say $f(x)=1-x$), you can build other ones using the following trick: let $\phi$ be an arbitrary invertible function, then let

$$g(x):=\phi^{-1}(f(\phi(x)))$$

With this definition, we have

$$g^2(x)=\phi^{-1}(f(\phi(\phi^{-1}(f(\phi(x))))))=\phi^{-1}(f(f(\phi(x))))=\phi^{-1}(\phi(x))=x.$$

So the space of solutions is huge.

I would expect the general equation to show similar richness, but I haven't found an easy way to adapt the trick.