0
$\begingroup$

I learning Standford CSS229 lecture note1.
There is a provement in 2.2 Least squares revisited as follows:
$\begin{align*} \nabla _\theta J(\theta) &= \nabla _\theta \frac{1}{2}(X\theta - \overrightarrow{y}) \\ &= \frac{1}{2} \nabla _\theta (\theta^TX^TX\theta - \theta^TX^T\overrightarrow{y} - \overrightarrow{y}^TX\theta+\overrightarrow{y}^T\overrightarrow{y}) \\ &= \frac{1}{2} \nabla _\theta tr(\theta^TX^TX\theta - \theta^TX^T\overrightarrow{y} - \overrightarrow{y}X^T\theta+\overrightarrow{y}^T\overrightarrow{y}) \\ &= \frac{1}{2} \nabla _\theta (tr\theta^TX^TX\theta - 2tr\overrightarrow{y}^TX\theta) \tag{$*$} \\ &= \frac{1}{2} (X^TX\theta + X^TX\theta - 2X^T\overrightarrow{y}) \tag{$**$} \\ &= X^TX\theta - X^T\overrightarrow{y} \end{align*}$
where $\nabla _\theta tr \ \overrightarrow{y}^TX\theta = X^T \overrightarrow{y}$ is to be derivated by $\nabla _A trAB = B^T$.
It seems that $A = \theta$, $B = \overrightarrow{y}^TX$.
But I think in this way the origin one equation should be $\nabla _\theta tr \ \theta\overrightarrow{y}^TX = X^T \overrightarrow{y}$
SO I don't know how to use this trace equation.
Could anyone help me?
Thanks in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.