8
$\begingroup$

Is it possible to revert the softmax function in order to obtain the original values $x_i$?

$$S_i=\frac{e^{x_i}}{\sum e^{x_i}} $$

In case of 3 input variables this problem boils down to finding $a$, $b$, $c$ given $x$, $y$ and $z$:

\begin{align} \frac{a}{a+b+c} &= x \\ \frac{b}{a+b+c} &= y \\ \frac{c}{a+b+c} &= z \end{align}

Is this problem solvable?

$\endgroup$
10
$\begingroup$

Note that in your three equations you must have $x+y+z=1$. The general solution to your three equations are $a=kx$, $b=ky$, and $c=kz$ where $k$ is any scalar.

So if you want to recover $x_i$ from $S_i$, you would note $\sum_i S_i = 1$ which gives the solution $x_i = \log (S_i) + c$ for all $i$, for some constant $c$.

$\endgroup$
  • $\begingroup$ So it’s solvable up to a constant. Thank you! $\endgroup$ – jojek May 18 '18 at 17:39
  • $\begingroup$ Which c constant should I use? There is any way of calculating it? $\endgroup$ – Joel Carneiro Feb 7 at 17:16
  • $\begingroup$ @JoelCarneiro Any $c$ will work; the solution is not unique. $\endgroup$ – angryavian Feb 7 at 17:58
2
$\begingroup$

A similar question was asked in a post of reddit. The answer below is adapted from that post:

$S_{i}$ = $\exp(x_{i})/(\sum_{i} \exp(x_{i}))$

Taking ln on both sides:

$\ln(S_{i}) = x_{i} - \ln(\sum_{i} \exp(x_{i}))$

Changing sides:

$x_{i} = \ln(S_{i}) + \ln(\sum_{i} \exp(x_{i}))$

The second term of the right hand side is constant for a particular $i$ and can be written as $C_{i}$. Therefore, we can write:

$x_{i} = \ln(S_{i}) + C_{i}$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.