I learning Standford CSS229 lecture note1.
There is a provement in 2.2 Least squares revisited as follows:
$\begin{align*} \nabla _\theta J(\theta) &= \nabla _\theta \frac{1}{2}(X\theta - \overrightarrow{y}) \\ &= \frac{1}{2} \nabla _\theta (\theta^TX^TX\theta - \theta^TX^T\overrightarrow{y} - \overrightarrow{y}^TX\theta+\overrightarrow{y}^T\overrightarrow{y}) \\ &= \frac{1}{2} \nabla _\theta tr(\theta^TX^TX\theta - \theta^TX^T\overrightarrow{y} - \overrightarrow{y}X^T\theta+\overrightarrow{y}^T\overrightarrow{y}) \\ &= \frac{1}{2} \nabla _\theta (tr\theta^TX^TX\theta - 2tr\overrightarrow{y}^TX\theta) \tag{$*$} \\ &= \frac{1}{2} (X^TX\theta + X^TX\theta - 2X^T\overrightarrow{y}) \tag{$**$} \\ &= X^TX\theta - X^T\overrightarrow{y} \end{align*}$
where $\nabla _\theta tr \ \overrightarrow{y}^TX\theta = X^T \overrightarrow{y}$ is to be derivated by $\nabla _A trAB = B^T$.
It seems that $A = \theta$, $B = \overrightarrow{y}^TX$.
But I think in this way the origin one equation should be $\nabla _\theta tr \ \theta\overrightarrow{y}^TX = X^T \overrightarrow{y}$
SO I don't know how to use this trace equation.
Could anyone help me?
Thanks in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.