# What is the derivative on $\theta_i$ given the function $CE$

Given a function $CE=-\sum_i y_i log(\hat{y}_i)$ and $\hat{y}_i=\frac{e^{\theta_i}}{\sum_j e^{\theta_j}}$, where $y$ and $\theta$ are vectors. The question asks to compute $\frac{\partial CE}{\partial \theta}$.

(Hint from the original question: $y$ is the one-hot label vector, you might want to consider the fact many elements of $y$ are zeros, and assume that only the $k$-th dimension of $y$ is one.)

My solution: $$\frac{\partial CE(y,\hat{y})}{\partial \theta_i}= -\sum_i y_i \frac{1}{\hat{y}_i} \frac{e^{\theta_i}\sum_j e^{\theta_i} -e^{2\theta_i}}{(\sum_j e^{\theta_j})^2}= -\sum_i y_i \frac{\sum_j e^{\theta_j}}{e^{\theta_i}} \frac{e^{\theta_i}\sum_j e^{\theta_j} -e^{2\theta_i}}{(\sum_j e^{\theta_j})^2}=\sum_i y_i (\hat{y}_i-y_i)$$

thus, $\frac{\partial CE(y,\hat{y})}{\partial \theta_i}$=\begin{cases} \hat{y}_i-1, \quad i=k\\ 0, \quad \text{otherwise} \end{cases}

Solution given:

$\frac{\partial CE(y,\hat{y})}{\partial \theta}=\hat{y}-y$

or equivalently, $\frac{\partial CE(y,\hat{y})}{\partial \theta_i}$=\begin{cases} \hat{y}_i-1, \quad i=k\\ \hat{y}_i, \quad \text{otherwise} \end{cases}

The difference shows on calculations of otherwise, did I miss anything in my solution?

Update (Solved):