This diagram is from Andrew Ng course for ML/DL:

enter image description here

But isn't the cost function (least squares function) shape depends on scatter of the data ?

For example below, the minimum will be at (0,1):

enter image description here

that doesn't correspond to convex shape (if you will imagine it in 3d plot), that Andrew Ng showed above.


Oh, i think I understand... my example is a convex shape too, but simply shifted by coordinates, relatively to the Andrew's example.

Am i right?

  • 1
    $\begingroup$ the cost function should be convex, which corresponds roughly to a "bow" shape $\endgroup$ – Tony S.F. Mar 8 '18 at 17:34
  • $\begingroup$ sorry, updating question $\endgroup$ – uptoyou Mar 8 '18 at 17:39

Because the cost function is given by:

$$ \frac{1}{2} \left\| X \theta - y \right\|_{2}^{2} $$

Which is a Linear Regression Problem with the Least Squares cost function which is a Convex Function of $ \theta $.

  • $\begingroup$ Can you please add more details to your answer? Thank you! $\endgroup$ – Alex Yursha Oct 10 '18 at 1:58
  • $\begingroup$ Could you point me what would you like to be extended? $\endgroup$ – Royi Oct 10 '18 at 14:32
  • $\begingroup$ The notation || is not familiar to me. Perhaps, at least provide links to supplemental resources of your choice, which will help to interpret the formula you wrote. Thank you! $\endgroup$ – Alex Yursha Oct 14 '18 at 5:52
  • $\begingroup$ The notation $ \left\| \cdot \right\| $ stands for norm. So the $ {\left\| \cdot \right\|}_{2} $ stands for the $ {L}_{2} $ norm and $ {\left\| \cdot \right\|}_{2}^{2} $ is the $ {L}_{2} $ norm squared. $\endgroup$ – Royi Oct 14 '18 at 6:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.