# A characterization of continuous nonincreasing functions

Let $f(x)$ be continuous in the interval $I := (0,1).$ Define $$D_+f(x_0) := \liminf_{h \to 0^+} \frac{f(x_0+h)-f(x_0)}{h}.$$ Put $$S:= \{x \in I: D_+ f(x) < 0\}.$$ Suppose that the set $f(I\backslash S)$ does not contain any non-empty open interval. Prove that $f(x)$ is non-increasing on $I.$

Argue by contrapositive.

Suppose that $f(a)<f(b)$ for some $a,b\in(0,1)$, $a<b$. For each $s\in (f(a),f(b))$ let $$x_s=\sup\{x\in (a,b):f(x)\le s\}$$ Observe that the set $\{x\in [a,b]:f(x)\le s\}$ is closed, nonempty and thus contains its supremum. Hence, $f(x_s)=s$. Also, since $f(x)>s$ for all $x\in (x_s,b)$, it follows that $D_+ f(x_s)\ge 0$. Thus, $x_s\in I\setminus S$. Conclusion: $(f(a),f(b))\subseteq f(I\setminus S)$.

The following might be an approach, although I'm not sure how to complete the argument.

We first wish to show that $D_+f(x) \leq 0$ for each $x \in I.$ Indeed, since $f(I\backslash S)$ is nowhere dense, we see $\big( f(I) \backslash f(I \backslash S)\big)^\circ$ is dense. Hence, there is a sequence $(x_n)$ with $D_+ f(x_n) \leq 0$ for each $n,$ such that $f(x) = \lim_{n \to \infty} f(x_n).$

I then wish to claim that $D_+ f(x) \leq \lim_{n \to \infty} D_+ f(x_n) \leq 0,$ but I can not write a rigorous proof for this.

If we were able to show the above claim, then it is easy to see that one can prove an analogue of the mean value theorem in terms of $D_+f(x):$ To be precise, let $x < y \in I.$ Then there exists $z \in (x,y)$ such that $\frac{f(y) - f(x)}{y-x} = D_+ f(z).$

With these two facts, the claim of the problem is established.

• Check the definition of nowhere dense. Your solution is not right. Commented Jan 1, 2014 at 16:45