# Partial Derivative with Respect to Vector Element

I'm looking through the Coursera course for Machine Learning and had a calculus question.

Let

$x^{(i)} = \begin{bmatrix}1 & x^{(i)}_1&x^{(i)}_2 & \cdots & x^{(i)}_p\end{bmatrix}$

$\theta = \begin{bmatrix}\theta_1 \\ \theta_2\ \\ \cdots \\ \theta_p \\ \theta_{p+1} \end{bmatrix}$

$y_i$ be a number.

I'm not sure I understand how to do this:

$\frac{\partial}{\partial \theta_j} y_i \theta x^{(i)} = y_ix_j^{(i)}$

When taking the derivative of $y_i \theta x^{(i)}$ with respect to $\theta_j$, it is makes some sense to me that the derivative is $y_ix_j^{(i)}$ as that $j$th change in theta is only multiplied against the $j$th item in $x^{(i)}$. However, I don't understand the mathematical way of deriving that.

• So $\theta x^{(i)}$ is a $(p+1)\times p$ matrix? – Alex Provost Dec 25 '17 at 16:52
• Yup, it was a mistake, I'll fix - thanks! :) – Bobby Dec 25 '17 at 17:03

\begin{align*} \frac{\partial}{\partial \theta_j} (y_i \theta x^{(i)}) & = y_i \frac{\partial}{\partial \theta_j} (\theta x^{(i)}) \\ & = y_i \frac{\partial}{\partial \theta_j} (\theta_1 x_1^{(i)} + \theta_2 x_2^{(i)} + \dotsb + \theta_p x_p^{(i)}) \\ & = y_i \left [\frac{\partial}{\partial \theta_j} (\theta_1 x_1^{(i)}) + \frac{\partial}{\partial \theta_j} (\theta_2 x_2^{(i)}) + \dotsb + \frac{\partial}{\partial \theta_j} (\theta_p x_p^{(i)}) \right] \\ & = y_i (0 + 0 + \dotsb + x_j^{(i)} + \dotsb + 0) \\ &= y_i x_j^{(i)}\end{align*}
• $\theta$ has $p+1$ entries? – Alex Ortiz Dec 25 '17 at 16:53
• Either I misunderstood the question or there is a mistake in the definitions of $x^{(i)}$ and $\theta$. I'm inclined to believe the latter, since $\frac{\partial}{\partial \theta_j} (y_i \theta x^{(i)})$ must equal $y_i x_j^{(i)}$ and not a $(p+1) \times p$ matrix, but I will edit or remove my answer if I'm wrong. – Luca Bressan Dec 25 '17 at 16:58
• Please, check your definitions one more time. Are you sure that $x^{(i)}$ is a row vector and $\theta$ a column vector and not the other way around? Moreover, there is now a problem in that the result should be $y_i x_{j-1}^{(i)}$ instead of $y_i x_j^{(i)}$. – Luca Bressan Dec 25 '17 at 17:11