I'm looking through the Coursera course for Machine Learning and had a calculus question.


$x^{(i)} = \begin{bmatrix}1 & x^{(i)}_1&x^{(i)}_2 & \cdots & x^{(i)}_p\end{bmatrix}$

$\theta = \begin{bmatrix}\theta_1 \\ \theta_2\ \\ \cdots \\ \theta_p \\ \theta_{p+1} \end{bmatrix}$

$y_i$ be a number.

I'm not sure I understand how to do this:

$\frac{\partial}{\partial \theta_j} y_i \theta x^{(i)} = y_ix_j^{(i)}$

When taking the derivative of $y_i \theta x^{(i)}$ with respect to $\theta_j$, it is makes some sense to me that the derivative is $y_ix_j^{(i)}$ as that $j$th change in theta is only multiplied against the $j$th item in $x^{(i)}$. However, I don't understand the mathematical way of deriving that.

  • 2
    $\begingroup$ So $\theta x^{(i)}$ is a $(p+1)\times p$ matrix? $\endgroup$ – Alex Provost Dec 25 '17 at 16:52
  • $\begingroup$ Yup, it was a mistake, I'll fix - thanks! :) $\endgroup$ – Bobby Dec 25 '17 at 17:03

$$\begin{align*} \frac{\partial}{\partial \theta_j} (y_i \theta x^{(i)}) & = y_i \frac{\partial}{\partial \theta_j} (\theta x^{(i)}) \\ & = y_i \frac{\partial}{\partial \theta_j} (\theta_1 x_1^{(i)} + \theta_2 x_2^{(i)} + \dotsb + \theta_p x_p^{(i)}) \\ & = y_i \left [\frac{\partial}{\partial \theta_j} (\theta_1 x_1^{(i)}) + \frac{\partial}{\partial \theta_j} (\theta_2 x_2^{(i)}) + \dotsb + \frac{\partial}{\partial \theta_j} (\theta_p x_p^{(i)}) \right] \\ & = y_i (0 + 0 + \dotsb + x_j^{(i)} + \dotsb + 0) \\ &= y_i x_j^{(i)}\end{align*}$$

  • $\begingroup$ $\theta$ has $p+1$ entries? $\endgroup$ – Alex Ortiz Dec 25 '17 at 16:53
  • $\begingroup$ Either I misunderstood the question or there is a mistake in the definitions of $x^{(i)}$ and $\theta$. I'm inclined to believe the latter, since $\frac{\partial}{\partial \theta_j} (y_i \theta x^{(i)})$ must equal $y_i x_j^{(i)}$ and not a $(p+1) \times p$ matrix, but I will edit or remove my answer if I'm wrong. $\endgroup$ – Luca Bressan Dec 25 '17 at 16:58
  • $\begingroup$ Fixed the definition in the question - thanks! :) $\endgroup$ – Bobby Dec 25 '17 at 17:04
  • $\begingroup$ And this makes a lot of sense - I'm now thinking it was a pretty bad questions. Thanks for your help!! :) $\endgroup$ – Bobby Dec 25 '17 at 17:06
  • $\begingroup$ Please, check your definitions one more time. Are you sure that $x^{(i)}$ is a row vector and $\theta$ a column vector and not the other way around? Moreover, there is now a problem in that the result should be $y_i x_{j-1}^{(i)}$ instead of $y_i x_j^{(i)}$. $\endgroup$ – Luca Bressan Dec 25 '17 at 17:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.