Can someone explain step by step how to to find the derivative of this softmax loss function/equation.
\begin{equation} L_i=-log(\frac{e^{f_{y_{i}}}}{\sum_j e^{f_j}}) = -f_{y_i} + log(\sum_j e^{f_j}) \end{equation}
where: \begin{equation} f = w_j*x_i \end{equation} let:
\begin{equation} p = \frac{e^{f_{y_{i}}}}{\sum_j e^{f_j}} \end{equation}
The code shows that the derivative of $L_i$ when $j = y_i$ is:
\begin{equation} (p-1) * x_i \end{equation}
and when $j \neq y_i$ the derivative is:
\begin{equation} p * x_i \end{equation}
It seems related to this this post, where the OP says the derivative of:
\begin{equation} p_j = \frac{e^{o_j}}{\sum_k e^{o_k}} \end{equation}
is:
\begin{equation} \frac{\partial p_j}{\partial o_i} = p_i(1 - p_i),\quad i = j \end{equation}
But I couldn't figure it out. I'm used to doing derivatives wrt to variables, but not familiar with doing derivatives wrt to indxes.