# Prove this property of softmax function?

Consider the softmax function $$f \colon \mathbb{R}^k \rightarrow \mathbb{R}^k$$ defined by $$f((x_1, \ldots, x_n)) = (y_1, \ldots, y_n)$$ where $$y_i = \frac{e^{x_i}}{\sum_j e^{x_j}}$$.

Can you show that for every $$x, \Delta y,\delta$$, there exists $$\epsilon$$ and $$\Delta x < \delta$$ such that $$f(x + \Delta x) = f(x) + \epsilon \cdot \Delta y$$? Also, is there a name for this property for general function? Assume there exists $$x'$$ such that $$y + \Delta y = f(x')$$.

I'm not so sure that you can make that statement. By the construction of the softmax, the sum of its entries is equal to $$1$$, in other words:

$$\sum_{i=1}^k [f(x + \Delta x)]_i = 1$$

and

$$\sum_{i=1}^k [f(x)]_i = 1$$

However, from what you've written,

$$f(x+\Delta x) = f(x) + \epsilon \Delta y$$

It would imply that

$$\sum_{i=1}^k [f(x) + \epsilon \Delta y]_i > 1$$

which leads us to a contradiction if we sum over the entries of $$f(x + \Delta x)$$:

$$\sum_{i=1}^k [f(x+\Delta x)]_i = \sum_{i=1}^k [f(x) + \epsilon \Delta y]_i > 1$$

• updated the question. I meant for any valid vector $\Delta y$. – listener Feb 1 at 5:39