# Nonlinear functions with an analytic solution for the zero set

For a linear transformation $$W$$, one can easily find an orthogonal projection matrix $$P$$ such that for all $$x$$, $$WPx=0$$. $$P$$ projects $$x$$ to the null space of $$W$$.

Assume a nonlinear transformation $$f(Wx)$$, where $$f$$ is some analytic nonlinear function that is almost everywhere differentiable (e.g., tanh or relu in the context of neural networks). Is there exists such a $$f$$ with a closed, analytic operation that "projects" each vector to its zero set?

Thanks!

• Do you mean to find $P$ such that $f(Px)=0$ for all $x$? Jan 12, 2020 at 8:26
• Yes, although $P$ can also be a nonlinear function. But crucially, I want $P$ to act analogously to orthogonal projection in the sense it projects to the closest zero - to prevent trivial solutions of e.g. mapping all x's to a negative number for relu. Jan 12, 2020 at 9:21
• So you want $p(x) \in f^{-1}(0), \forall x$ and $\lVert p(x) - x \rVert$ is minimized. I doubt you can find a $p$ for a generic $f$. In linear case, you need to find a basis for $W$ and similarly first you need to characterize $f^{-1}(0)$. For example, for single variable relu function $p(x)=\begin{cases}0 & x > 0 \\ x & x \leq 0\end{cases}$. Jan 14, 2020 at 13:10
• Thanks for the comment. I don't want to find a $P$ for every $f$; finding such $P$ for a specific "well behaved" (differentiable, analytic) nonlinear $f$ would be enough. Jan 14, 2020 at 13:29