1
$\begingroup$

enter image description here

I'm not sure how I should tackle such problems. To my understanding, $\omega_i$ shows the strength/importance of this edge in the prediction of $f(x)$. But I'm not sure what the gradient with respect to $\omega_i$ is an indication of?

What would the outcome of this problem be if the derivatives all had different values ?

$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

These partial derivatives appear for example in error back propagation.

If we are doing supervised learning, we can feed forward through the network, then measure the error and use the vector chain rule to derive or "back-propagate" the gradient of this error throughout the network. So we can with the help of chain rule for differentials do kind of like a gradient-descent optimization to fit the neural network to the training data.

$\endgroup$
2
  • $\begingroup$ Yes for back propagation we use the gradient of the cost function to change the weights and in that case, to my understanding, the value of the partial derivative with respect to a specific $\omega$ is representative of how much a change in its value is going to influence the cost (i.e. which weights are the most important). But I fail to see how it can help me understand the exercise. I know it's supposed to be easy but I'm still confused $\endgroup$
    – ncmq
    Jan 13, 2020 at 12:24
  • 1
    $\begingroup$ @ncmq : I can also not solve this particular one. It is a really uninteresting toy network. My brain just shuts down at such meaningless exercises. I'm sorry. $\endgroup$
    – mathreadler
    Jan 13, 2020 at 12:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .