Solving the equation with sum of max functions (ReLU) Given real constants $y$, $w_1$...$w_n$, $b_1$...$b_n$, is it possible to solve the unknown $x$ in closed form in the below equation?
$$
y = \sum_i \max\{w_ix + b_i, 0\}
$$
 A: You can start using that:
$$ \max\{x,y\}=\frac{x+y+\vert x-y \vert}{2}.$$
In particular, you have that:
$$ \max\{x,0\}=\frac{x+\vert{x}\vert}{2} $$
Then $$\displaystyle{y=\sum_{i=1}^{n}\frac{w_{i}x+b_{i}+\vert{w_{i}x+b_{i}}\vert}{2}}$$
You need to observe that for each $i$ you have two cases:
$$\frac{w_{i}x+b_{i}+\vert{w_{i}x+b_{i}}\vert}{2}=\begin{cases} w_{i}x+b_{i} & \text{if}\ \ w_{i}x+b_{i}\geq 0, \\ \quad 0 & \text{if}\ \ w_{i}x+b_{i}< 0.\end{cases}$$
Because:
$$ w_{i}x+b_{i}<0 \Rightarrow \vert{w_{i}x+b_{i}}\vert=-(w_{i}x+b_{i})
\Rightarrow \frac{w_{i}x+b_{i}+\vert{w_{i}x+b_{i}}\vert}{2}=0 $$
and
$$ w_{i}x+b_{i}\geq 0 \Rightarrow \vert{w_{i}x+b_{i}}\vert=w_{i}x+b_{i}
\Rightarrow \frac{w_{i}x+b_{i}+\vert{w_{i}x+b_{i}}\vert}{2}=w_{i}x+b_{i} $$
Then your sum reduces to:
$$y=\sum_{i\,\vert\ w_{i}x+b_{i}\geq 0}\left(w_{i}x+b_{i}\right).$$
since $x$ does not depend on $i$ you can write the last equation like that:
$$ y =\left(\sum_{i\,\vert\ w_{i}x+b_{i}\geq 0}w_{i}\right)x+\left(\sum_{i\,\vert\ w_{i}x+b_{i}\geq 0}b_{i}\right)$$
From there you can isolate $x$ at least when the sum $\displaystyle{\sum_{i\,\vert\ w_{i}x+b_{i}\geq 0}w_{i}}$ is not zero. In fact:
$$ x = \frac{\displaystyle{y-\sum_{\{i\,\vert\ w_{i}x+b_{i}\geq 0\}}b_{i}}}{\displaystyle{\sum_{\{i\,\vert\ w_{i}x+b_{i}\geq 0\}}w_{i}}}.$$
A: Or you can simply say:
$$
y = \sum_{i=1}^{n}\max\{w_ix + b_i, 0\} = \sum_{i\,\vert\, w_{i}x+b_{i}\geq 0}(w_{i}x + b_{i})=\left(\sum_{i\,\vert\, w_{i}x+b_{i}\geq 0}w_{i}\right)x+\left(\sum_{i\,\vert\, w_{i}x+b_{i}\geq 0}b_{i}\right).
$$
because, for each $k$ such that $\ w_{k}x+b_{k}<0\ $ the value of  $\ \max\{w_{k}x + b_{k}, 0\}\ $ vanish. That's why we have:
$$ \sum_{i=1}^{n}\max\{w_ix + b_i, 0\} = \sum_{i\,\vert\, w_{i}x+b_{i}\geq 0}(w_{i}x + b_{i}). $$
