Problems with indices and chain rule Is there way to handle $\sum_i\frac{\partial}{\partial p_j}p_i\ln p_i$ with the Einstein summation convention without getting troubles with indices?
$\frac{\partial}{\partial p_j}p_i\ln p_i=\delta_{ij}\ln p_i+\frac{p_i}{p_i}\delta_{ij}$
 A: The Einstein summation convention is that any index which is repeated implies a summation over that index. For example
$$\eqalign{
p_i(\log p_i) \:&\equiv\: \sum_i p_i(\log p_i) \\
}$$
Extending the Kronecker delta symbol to three indexes
$$\eqalign{
\def\LR#1{\left(#1\right)}
\def\d{\vec\delta}
\def\o{{\tt1}}
\def\p{\partial}
\def\qiq{\quad\implies\quad}
\def\grad#1#2{\frac{\p #1}{\p #2}}
\def\gradLR#1#2{\LR{\grad{#1}{#2}}}
\d_{ijk}
 &= \begin{cases}
  1 \quad&{\rm if\;} i=j=k \\
  0 \quad&{\rm otherwise} \\
\end{cases} \\
}$$
allows you to write the elementwise/Hadamard product of vectors using Einstein notation
$$\eqalign{
c = a\odot b \qiq c_j = a_i\,\d_{ijk}\,b_k \\
}$$
If $q$ is the Hadamard inverse of $p,\,$ then it satisfies
$$\eqalign{
\o = p\odot q \qiq \o_j = p_i\,\d_{ijk}\,q_k \\
}$$
and it can be used to write the elementwise derivative of $\log(p)$
in index notation
$$\eqalign{
\p_j(\log p_i) &= \d_{jik}\,q_k \\
}$$
Putting it all together
$$\eqalign{
\p_j\LR{p_i\log(p_i)}
 &= \LR{\p_jp_i}\log(p_i) + p_i \p_j\log(p_i) \\
 &= \delta_{ij}\log(p_i) + p_i \d_{jik}\,q_k \\
 &= \log(p_j) + \o_j \\
}$$
A: $\frac{\partial}{\partial p_j}(p_i\ln p_i)=\delta_{ij}(\ln p_i+\frac{p_i}{p_j})=0$ for all $i\neq j$, so we only need to consider the $j$-th term, which is $\ln p_j+1$.
