1
$\begingroup$

NB: I'm not a Stanford student asking for you to do my job, I'm studying online and trying to figure it out.

I'm following Stanford's CS224D and I'm having troubles understanding some of the math.

In the first assignment's solutions, part II question c, I don't understand this part:

Derive the gradients with respect to the inputs $x$ to an one-hidden-layer neural network (that is, find $∂J/ ∂x$ where $J$ is the cost function for the neural network). The neural network employs sigmoid activation function for the hidden layer, and softmax for the output layer. Assume the one-hot label vector is $y$, and cross entropy cost is used.

The starting point is this: finding

$$\frac{∂(CE)}{ ∂x}$$

Where CE is the the cross entropy function.

enter image description here

I understand that the result is obtained using the chain rule, but I can't see which function is $f$ and which function is $g$ [if we describe the chain rule for a function $f(g(x))$].

Could anyone help me figure how to apply the chain rule to this case ?

Any input/help will be much appreciated.

$\endgroup$

1 Answer 1

0
$\begingroup$

Considering only a single term in the sum, I think you have $$ f_i(g_i(\theta_i))=-y_i\log(\hat y_i) $$ so that $f_i(s)=-y_i\log(s)$ and $g_i(t)=\operatorname{softmax}_i(t)$. Does that make sense?

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .