If $f \circ f =f$, then what is $f$? This is an exercise from Mathematical Analysis II by Zorich:(the surface here is the same as manifold, not necessarily 2-dimensional.)

Let $f:\mathbb R^n
 \to \mathbb R^n$
be a smooth mapping satisfying the condition $f \circ f =f$.
a) Show that the set $f(\mathbb R^n)$ is a smooth surface in $\mathbb R^n$.
b) By what property of the mapping $f$ is the dimension of this surface determined?

My solution:
$$f'(f(x))f'(x)=f'(x)\Rightarrow \forall x\in f(\mathbb R^n),f(x)=x,(f'(x)-I)f'(x)=O.\tag1$$ We can conclude that eigenvalue of $f'(x)$ is $0$ or $1$. Let $P^{-1}(x)f'(x)P(x)=J(0)+\begin{pmatrix}I_r&O\\O&O\end{pmatrix}$where $|P(x)|\neq0$, and according to (1) we can know that $J(0)=O$. Since $tr(f'(x))=rank(f'(x))$ is continous on $f(\mathbb R^n)$, and $f(\mathbb R^n)$ is connected, we can know that $\forall x\in f(\mathbb{R}^n), rank(f(x))\equiv r$.
$\forall x \in \mathbb{R}^n, (P^{-1}(x)\begin{pmatrix}I_r&O\\O&O\end{pmatrix}P(x)-I)f'(x)=O\Rightarrow rank(f'(x))\leq r$
$\forall f(x_0)\in f(\mathbb{R}^n),\ rank (f'(f(x_0)))=r, \forall x\in O(f(x_0)),rank(f'(x))\leq r\Rightarrow \exists \bar O(f(x_0)),\forall x\in \bar O(f(x_0)), rank(f'(x))=r$. According to The rank theroem, we can find $\tilde O(x_0), \tilde O(f(x_0))$ and diffeomorphisms $\phi ,\varphi,f=\phi\circ\pi\circ\varphi^{-1}$ where $ \pi:(u^1,\cdots,u^r,\cdots,u^m)\mapsto(u^1,\cdots,u^r)$, so that the set $f(\mathbb R^n)$ is a smooth surface in $\mathbb R^n$ and the dimension depends on the rank of $f$ on $f(\mathbb R^n)$.
My doubts:
(1) Is my solution above correct? Feel free to point out any mistakes or offer any concise solutions.
(2)Does $f$ have to be a linear mapping with the form $f(x)=P^{-1}\begin{pmatrix}I_r&O\\O&O\end{pmatrix}P$$\cdot$$x$?
(3)If we only assume $f\in C^0(R^m,R^m)$, what properties of $f$ can we get? Is $f(\mathbb R^n)$ still a surface?
 A: Your solution has all the correct ideas but it is written quite sloppily. For example:

*

*You say that the eigenvalues of $f'(x)$ (where $x \in \operatorname{im}(f)$) are $0$ or $1$. However, this is not enough to justify that $f'(x)$ is diagonalizable. The key property is that the matrix $A = f'(x)$ satisfies $A^2 = A$ and so $A$ is a projection matrix, hence diagonalizable with eigenvalues $0,1$.

*For some reason you write
$$ P(x)^{-1} \cdot f'(x) \cdot P(x) = J(0) + \begin{pmatrix} I_r & O \\ O & O  \end{pmatrix}. $$ It is not clear what $J(0)$ is and what you are doing. If you just diagonalize $f'(x)$ then there is no $J(0)$, you don't need to argue that it is $0$. Also, it is better to use $0$ (or even better, $0_{r \times (n-r)}, 0_{(n-r) \times r}$, etc) instead of $O$ as you use $O$ later to denote open sets.

*You want to justify that the rank of $f'(x)$ is $\leq r$ over all $\mathbb{R}^n$. The correct equation to justify this is
$$ \left( \underbrace{P(f(x)) \cdot \begin{pmatrix} I_r & 0 \\ 0 & 0 \end{pmatrix} \cdot P^{-1}(f(x))}_{f'(f(x))} - I_n \right) \cdot f'(x) = 0 $$
and not what you wrote. Note the different argument for $P$ and the reverse of roles of $P, P^{-1}$.

*You conclude that each point $x_0 \in \operatorname{im}(f)$ has an open neighborhood $O$ in $\mathbb{R}^n$ such that $f'(x)$ has constant rank on $O$ but you don't really justify it.

*You use the constant rank theorem but write the wrong conclusion (which is that $f$ looks like $(u^1, \dots, u^n) \rightarrow (u^1, \dots, u^r, 0, \dots, 0)$) and then don't say why this implies that $\operatorname{im}(f)$ is a surface.

Regarding your other questions:

*

*No, $f$ doesn't have to be linear (nor affine). Note that if $\varphi \colon \mathbb{R}^n \rightarrow \mathbb{R}^n$ is a diffeomorphism, then if $f$ satisfies $f \circ f = f$ then $h = \varphi^{-1} \circ f \circ \varphi$ also satisfies $h \circ h = h$. So you can take a projection and conjugate it by a diffeomorphism to obtain (usually) a non-linear example. For a concrete example, take $f(x,y) = (0,y)$ and $\varphi(x,y) = (x, y + x^2)$. Then
$$ h(x,y) = \left( \varphi^{-1} \circ f \circ \varphi \right)(x,y) = (0,y + x^2) $$
is a non-linear map which satisfies $h \circ h = h$ as you can readily verify directly.

*No, $\operatorname{im}(f)$ won't necessarily be a smooth manifold or even a topological manifold. For an example where $\operatorname{im}(f)$ is not a smooth manifold, take $f \colon \mathbb{R} \rightarrow \mathbb{R}$ to be $f(x) = |x|$. Then $\operatorname{im}(f) = [0,\infty)$ which is not smooth manifold (without boundary). For an example where $\operatorname{im}(f)$ is not even a manifold, take $f \colon \mathbb{R}^2 \rightarrow \mathbb{R}^2$ which is defined by
$$ f(x) = \left( \operatorname{sgn}(x) \cdot (|x| - \min(|x|,|y|), \operatorname{sgn}(y) \cdot (|y| - \min(|x|,|y|) \right). $$
The map $f$ is continuous, satisfies $f \circ f = f$ with image the union of the $x$ and $y$ axes in $\mathbb{R}^2$.

A: Question: "Let $f:R^n→R^n$ be a smooth mapping satisfying the condition $f∘f=f$. a) Show that the set $f(R^n)$ is a smooth surface in $R^n$. b) By what property of the mapping $f$ is the dimension of this surface determined?"
Answer: Most books in differential geometry define a "surface" to be a smooth manifold of dimension $2$ and it seems your claim is not correct. Let $U:=\mathbb{R}^n:=V_1\oplus V_2$ be a direct sum with $V_i \cong \mathbb{R}^{n_i}$ with $n_1 \neq 2$. Let the map $f:U \rightarrow U$ be defined as $f(v):=f(v_1,v_2):=(v_1,0)$. The map $f$ is a linear map, hence it is smooth and $Im(f)=V_1$ which has dimension $n_1\neq 2$. You will find books in differential geometry proving that affine real space $\mathbb{R}^n$ is a smooth manifold of dimension $n$.
