$f\circ g(x)=(x-1)$ and $g\circ f(x)=(x+1).$ Can we prove $f$ and $g$ are linear functions? $f\circ g(x)=(x-1)$ and $g\circ f(x)=(x+1).$ Can we prove $f$ and $g$ are linear functions?
In my attempt to solve a question I encountered different cases, One case is this question. Can we really say $f$ and $g$ are linear? I have a feeling that they are not.
 A: Below is an example of non-linear functions. Let $A, B$ are 
any rational numbers, $A \ne B$. Let \begin{equation}
g(x) = 
 \begin{cases}
   A-x &\text{if $x\in \mathbb{Q}$}\\
   B-x &\text{if $x\in \mathbb{I}$}
 \end{cases}
\end{equation}
\begin{equation}
f(x) = 
 \begin{cases}
   A-x-1 &\text{if $x\in \mathbb{Q}$}\\
   B-x-1 &\text{if $x\in \mathbb{I}$}
 \end{cases}
\end{equation}
It easy to see that $f(g(x))=x-1$, $g(f(x))=x+1$.
A: Suppose that $f,g$ are continuous. Then $f,g$ are linear.

For let $f(x)=-x-a(-x)$, $g(x)=1-x-b(-x)$. Calculating $f\circ g \circ f$, $g\circ f\circ g$ in 
two different ways we see that $a$ and $b$ have unit period, and the given relations rewrite to
$$\begin{align}
a(x)&=b(x+a(x))\\
b(x)&=a(x+b(x))
\end{align}$$
Additionally, $a,b$ are continuous, and each is constant iff the other is. 
So suppose $a$ were not constant. Then by the intermediate value theorem, $a$ would take on an irrational value $y$ at some $x$. For this $x$, we would have
$$y=a(x)=b(x+y)=a(x+2y)=b(x+3y)=\ldots\text{,}$$
and by density of fractional parts of multiples of a given irrational on the unit interval, $a$ and $b$ would equal $y$ on a dense set. By continuity, they would then be constant functions - contradiction. Therefore $a,b$ are constant, so $f,g$ are linear.
A: We have that $f(g(x)) = x-1$ and $g(f(x)) = x+1$. We can take $g(f(g(x))) = g(x-1) = g(x)+1$. I got these relations by plugging in $g(x)$ into $g(f(x))$ and by taking having $f(g(x))$ as the argument in $g(x)$. Similarly, we have that $f(g(f(x))) = f(x)-1 = f(x+1)$
Since $g(x-1) = g(x)+1$, we get that $g(x) = g(x-1) - 1$. This means that $g(x) = -x+g_0(\{x\})$, where $g_0(x)$ is a function from $0\le x < 1$ and $\{x\}$ is the fractional part of $x$.
Similarly, we have that $f(x+1) = f(x)-1$, meaning $f(x) = -x + f_0(\{x\})$.
Going from the starting relations, we have that $f(g(x)) = x-1$, meaning that $$x-g_0(\{x\}) + f_0(\{-x+g_0(\{x\})\}) = x-1$$
$$g_0(\{x\}) - f_0(\{-x+g_0(\{x\})\}) = 1$$
Similarly, we have that $$x-f_0(\{x\}) + g_0(\{ -x+f_0(\{x\}) \}) = x+1$$
$$-f_0(\{x\}) + g_0(\{ -x+f_0(\{x\}) \}) = 1$$
Both relations must be simplified for $f$ and $g$ to be valid functions. In the case of only integers, this simplifies to $f(x) = -x+f_0$ and $g(x) = -x + f_0+1$, which are both linear. However, any other function may not be linear (an example is given in the other question).
