# How to prove the non-existence of continuous solution for a functional equation?

Let us consider the following functional equation $$f(f(x))=-x^3+\sin(x^2+\ln(1+|x|))$$ How to prove that there is NO continuous function $$f:\mathbb{R}\to\mathbb{R}$$ satisfies the equation above?

First I notice that $$F(x):=-x^3+\sin(x^2+\ln(1+|x|))$$ has only one fixed point $$0$$. From this it follows that $$f(0)=0.$$ Otherwise, if we assume that $$a:=f(0)>0$$, then $$f(a)=F(0)=0$$. It is easy to see that $$f(x)$$ has a fixed point in $$(0,a)$$ by intermediate value theorem. Similarly, we can rule out the case $$f(0)<0$$. Therefore we have $$f(0)=0$$

I want to show that $$f$$ has a nonzero fixed point $$x^*$$, from which we find a contradiction. This is my initial idea. But I don't how to find such non-zero fixed point.

If I find a closed interval $$[a,b]$$, which does not include 0, such that $$f([a,b])\subset [a,b]$$. Then we know that $$f$$ has at least one fixed point in $$[a,b]$$ by intermediate value theorem.

Maybe, my idea is wrong. Are there some other approaches?

What follows is the plot of function $$F$$, which may give you some hints:

Another approach: look to large values. Let $$g(x)$$ be that expression on the right, and note that $$\lim_{x\to\infty} g(x)=-\infty$$ and $$\lim_{x\to-\infty} g(x)=\infty$$. Also, there is some $$M$$ such that $$g$$ is strictly decreasing on both $$(-\infty,-M)$$ and $$(M,\infty)$$. Suppose there is a continuous $$f$$ with $$f\circ f = g$$. Then, since $$g$$ is injective on $$(-\infty,-M)$$, $$f$$ is also injective on $$(-\infty,-M)$$, hence monotone. Let $$L=\lim_{x\to\infty} f(x)$$, in the extended real line. If $$L$$ is finite, then by continuity $$f(L)=\lim_{x\to\infty}f(f(x))=-\infty$$. That's impossible, so either $$L=\infty$$ or $$L=-\infty$$. If $$L=\infty$$, then $$\lim_{x\to\infty} f(f(x)) =\lim_{y\to\infty} f(y) = \infty$$, a contradiction. If $$L=-\infty$$, then $$-\infty = \lim_{x\to\infty} f(f(x))=\lim_{y\to-\infty} f(y)$$, and we'll have the same problem on the other side ($$\lim_{x\to-\infty} f(f(x))=\lim_{y\to\infty} f(y)=-\infty$$). Either way, we can't have both $$\lim_{x\to\infty} f(f(x))=-\infty$$ and $$\lim_{x\to-\infty} f(f(x))=\infty$$ on the same globally continuous $$f$$. There is no such function.