# Invert the softmax function

Is it possible to revert the softmax function in order to obtain the original values $x_i$?

$$S_i=\frac{e^{x_i}}{\sum e^{x_i}}$$

In case of 3 input variables this problem boils down to finding $a$, $b$, $c$ given $x$, $y$ and $z$:

\begin{align} \frac{a}{a+b+c} &= x \\ \frac{b}{a+b+c} &= y \\ \frac{c}{a+b+c} &= z \end{align}

Is this problem solvable?

## 2 Answers

Note that in your three equations you must have $x+y+z=1$. The general solution to your three equations are $a=kx$, $b=ky$, and $c=kz$ where $k$ is any scalar.

So if you want to recover $x_i$ from $S_i$, you would note $\sum_i S_i = 1$ which gives the solution $x_i = \log (S_i) + c$ for all $i$, for some constant $c$.

• So it’s solvable up to a constant. Thank you! – jojek May 18 '18 at 17:39
• Which c constant should I use? There is any way of calculating it? – Joel Carneiro Feb 7 at 17:16
• @JoelCarneiro Any $c$ will work; the solution is not unique. – angryavian Feb 7 at 17:58

A similar question was asked in a post of reddit. The answer below is adapted from that post:

$$S_{i}$$ = $$\exp(x_{i})/(\sum_{i} \exp(x_{i}))$$

Taking ln on both sides:

$$\ln(S_{i}) = x_{i} - \ln(\sum_{i} \exp(x_{i}))$$

Changing sides:

$$x_{i} = \ln(S_{i}) + \ln(\sum_{i} \exp(x_{i}))$$

The second term of the right hand side is constant for a particular $$i$$ and can be written as $$C_{i}$$. Therefore, we can write:

$$x_{i} = \ln(S_{i}) + C_{i}$$