Are the only solutions to this implicit functional equation linear? Let $f:(0,1] \to (-\infty,0]$ be a smooth function which is strictly negative on $(0,1)$ and satisfies $f(1)=0$.
Let $\epsilon \in (0,1)$ and let $x,y:(0,\epsilon) \to (0,1]$ be smooth functions satisfying $x(s)y(s)=s$, $x(s) < y(s)$ for every $s \in (0,\epsilon)$.
Suppose that $f \circ x=- y, f \circ y=- x$, i.e.
$$
f\big(x(s)\big)=- y(s),f\big(y(s)\big)=- x(s) \tag{1} \label{1}
$$
for every $s \in (0,\epsilon)$.

Is it true that $f(x)=x-1$?


In the case where $f(x)=x-1$, equation \eqref{1} implies $x(s)+y(s)=1$, which together with $x(s)y(s)=s$, implies that
$$x(s)=\frac{1-\sqrt{1-4s}}{2}\text,y(s)=\frac{1+\sqrt{1-4s}}{2}\text,$$
so $x,y$ are defined on $\left(0,\frac{1}{4}\right]$.
The motivation for this question comes from this optimization problem.
Comment:
I specifically assumed a strict inequality $x<y$. This is important, since otherwise we may get more solutions: Set $x(s)=y(s)=\sqrt s$ and $f(x)=-x$.
(Although $f(x)=-x$ does not satisfy $f(1)=0$, so it needs to be modified after $s=\epsilon$ in order to satisfy this boundary condition as well).
 A: That's far from true. The following construction gives an uncountable family of counterexamples.
For a given $ \epsilon \in ( 0 , 1 ) $, choose $ \alpha \in \left( 1 , \frac 1 { \sqrt \epsilon } \right) $ and take any smooth $ x : ( 0 , \epsilon ) \to ( 0 , 1 ] $ such that:

*

*$ s \mapsto \frac s { x ( s ) } $ is strictly decreasing;

*$ c = \lim _ { s \to 0 ^ + } \frac s { x ( s ) } < 1 $;

*$ s \le x ( s ) \le \frac s { \alpha \sqrt \epsilon } $ for every $ s \in ( 0 , \epsilon ) $.

Define $ y : ( 0 , \epsilon ) \to ( 0 , 1 ] $ by $ y ( s ) = \frac s { x ( s ) } $. Then we'll have the following:

*

*$ x $ is strictly increasing;

*$ x $ and $ y $ are local diffeomorphisms (by the inverse function theorem);

*$ x ( s ) y ( s ) = s $ for every $ s \in ( 0 , \epsilon ) $;

*there is $ a \in \left( 0 , \frac { \sqrt \epsilon } \alpha \right] $ such that $ \operatorname { ran } x = ( 0 , a ) $;

*There is $ b \in \left[ \alpha \sqrt \epsilon , c \right) $ such that $ \operatorname { ran } y = ( b , c ) $;

*$ x ( s ) < y \left( s ' \right) $ for every $ s , s ' \in ( 0 , \epsilon ) $.

Now define:
$$
\begin {cases}
g : ( 0 , a ) \to ( - \infty , 0 ] \\
g ( t ) = - y \left( x ^ { - 1 } ( t ) \right)
\end {cases}
\qquad
\begin {cases}
h : ( b , c ) \to ( - \infty , 0 ] \\
h ( t ) = - x \left( y ^ { - 1 } ( t ) \right)
\end {cases}
$$
Then $ g $ and $ h $ are smooth and strictly increasing functions, for every $ t \in ( 0 , a ) $ we have $ - c < g ( t ) < - b $, and for every $ t \in ( b , c ) $ we have $ - a < h ( t ) < 0 $. Thus we can mutually extend $ g $ and $ h $ to a strictly increasing and smooth $ f : ( 0 , 1 ] \to ( - \infty , 0 ] $ with $ f ( 1 ) = 0 $, by choosing suitable smooth functions filling $ [ a , b ] $ and $ [ c , 1 ] $. Then $ f $ will satisfy all the desired properties. As we are very much free in choosing $ x $ and extending $ g $ and $ h $, we end up constructing an uncountably infinite family of such functions.
EDIT:
I would like to add that in situations like this, where there is a very large family of smooth functions satisfying certain properties, it may be useful to search for analytic functions in that family. In many cases, that may give a unique solution, which you are seeking. Analytic functions are very rigid and that makes them very rare among smooth functions. For example, the above construction is based on continuing a smooth function in a much free manner. That can't be done for analytic functions, as the analytic continuation is unique. It's interesting that the functions $ x $, $ y $ and $ f $ given by yourself are analytic on their domains. At the moment, I don't know whether that's the only analytic possible case or not.
