# Why Softmax function can make output values sum up to 1

In mathematics, the softmax function, or normalized exponential function, is a generalization of the logistic function that "squashes" a $K$-dimensional vector $z$ of arbitrary real values to a $K$-dimensional vector $\sigma(z)$ of real values in the range $(0, 1)$ that add up to $1$. The function is given by

$$\sigma(z)_j=\frac{e^{z_j}}{\sum_{k=1}^Ke^{z_k}},\quad \text{for }j=1,...,K$$

I just can't understand why this softmax equation can keep result values sum up to 1.

From M. Winter's answer, I know zj =zk, so the answer is simple. I feel sometimes hard to understand the English style explanation of some equations.

This one using j and k is confused, which make I think ezk is the previous layer output, and ezj is current new output. There is many equations contain such writing, some time j and k is diferent , just say both j and k are K-dimensional is not enough.

### And ,In my opinion:

For example: if input X= [1,2,3,4,1,2,3], assign each element as Xi (i from 0 to X length 6). Why not just calculate Xi / SUM(X1 +.... X6) for each element ? e seems meanless here, which also a factor make me think ezk and ezj are different.

Try computing the sum of the components of $\sigma(z)$, i.e. $\sum_{j=1}^K \sigma(z)_j$. You will see
• Using j and k is confused, which make I think ezk is the previous layer output, and ezj is current new output. There is many equations contain such writing, some time j and k is diferent , just say both j and k are K-dimensional is not enough. – Mithril May 6 '17 at 3:30
• And for example, if input X= [1,2,3,4,1,2,3], assign each element as Xi (i from 0 to X length 6). Why not just calculate Xi / SUM(X1 +.... X6) for each element . e seems meanless, which also a factor make me think ezk and ezj are different. – Mithril May 6 '17 at 3:40
• @Mithril Suppose $K=3$. Then $\frac{\sum_{j=1}^K e^{z_j}}{\sum_{k=1}^K e^{z_k}}=\frac{e^{z_1}+e^{z_2}+e^{z_3}}{e^{z_1}+e^{z_2}+e^{z_3}}=1$. The fact that you use different letters to index the two sums doesn't mean they have different values. I think in general you may be a bit confused about summation notation. – Ian May 6 '17 at 3:55
• @Mithril The $e$ is there to make the answer positive. Without this, the $\sigma(z)_j$ could be a bunch of numbers, still adding up to 1, but also negative, like $-1000$ and $1001$. This is unwanted and you want results in $[0,1]$. And no matter what you plug in into $e^z$, the resuklt is always positive. So this ensures the $\sigma(z)_j>0$ part. The second $\sigma(z)_j<1$ part is ensured as described in my answer. Also I can't see how I confused $j$ and $k$, which does not matter anyway as they are just indices. – M. Winter May 6 '17 at 10:30