Consider a training set $\{(\vec{x^{(n)}}, \vec{t^{(n)}}) \in \rm I\!R^D \times I\!R^K : n = 1, ..., N \}$ where :

  • $\vec{t^{(n)}}$ is an indicator for the class membership of $\vec{x^{(n)}}$, i.e $\vec{t^{(n)}} = (0, 1, ... 0)$ if $\vec{x^{(n)}}$ belongs to $C2$

  • X is a $N \times (D+1)$ matrix whose nth row is $(\vec{x^{(n)}})^T$

  • T is a $N \times K$ matrix whose nth row is $(\vec{t^{(n)}})^T$

The least square problem is to minimise the objective function:

$G($W$)$ = $\frac{1}{2}\sum_{n=1}^{N}\sum_{k=1}^{K}[y_k(\vec{t^{(n)}};\vec{w_k}) - t_k^{(n)}]^2$

where $y_k(\vec{t^{(n)}};\vec{w_k})$ is the linear discriminant function.

I know that W is minimised by W = (X$^T$X)$^{-1}$X$^T$X

However, I don't understand why there is this $\frac{1}{2}$ coefficient in front of the double summation.

  • 1
    $\begingroup$ Does the factor $1/2$ matter? Whichever $W$ minimizes $G(W)$ also minimizes $2G(W)$. $\endgroup$ – Wouter Feb 13 '18 at 13:52

It's a simplification tool. When you take the derivative of the objective function and the exponent $2$ comes out to multiply the double sum it will be cancelled out. Similarly, when you set the objective function's derivative to $0$ you could just as easily divide out the $2$ in the front.

  • $\begingroup$ Ok, it makes sense, thank you. $\endgroup$ – Pierre P. Feb 13 '18 at 13:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.