I am studying machine learning and I am a bit rusty on deriving complex derivatives. I have taken Andrew Ng's course on machine learning and seen that the derivative of the cost function with respect to $\theta$ is as follows.

$$ cost = J(\theta) = \sum_{i=1}^m - log \space p(y_i|x_i; \theta) $$

where $ h_\theta(x_i) $ is $ \theta_1x_1 + \theta_2x_2 + ... $ and the summation term $ log \space p(...) $ is

$$ -y_i \space log(h_\theta(x_i) - (1 - y_i) \space log(1 - h_\theta(x_i)) $$

and the derivative turns out to be

$$ \frac{\partial}{\partial \theta_j} = \frac{1}{m} \sum_{i=1}^m (h_\theta(x_i) - y_i)x_j $$

But I have now come across a new situation where $\theta$ is actually destructured into different components such that the cost function is as follows with $ \theta = \sigma + PQ $...

$$ J(\theta) = \frac{1}{m} \sum_{i=1}^m log \space p(y_i | x_i; \{ \sigma + ab \}) $$

I think I need to optimize for the $a$ and $b$ variables first and then only optimize $\sigma$ as the last resort, but I am getting stuck on how to take the partial derivative with respect to $a$, $b$, or $\sigma$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.