0
$\begingroup$

I was trying to implement linear regression with gradient descent using the equation presented on the machine learning course on Coursera:

$$\Theta_{j}:=\Theta_{j}-\alpha\frac{1}{m}\sum_{i=1}^{m}{(h_{\Theta}(x^{(i)})-y^{(i)})*x^{(i)}_{j}}$$

However when I run it doesn't converge to the real values:

figures

I guess I implemented something wrong, I just cannot find it. I was looking for other papers on the internet and found different implementations for gradient descent, such as:

$$\Theta_{j}:=\Theta_{j}+\alpha\sum_{i=1}^{m}{(y^{(i)}-h_{\Theta}(x^{(i)}))*x^{(i)}_{j}}$$

(found at: http://cs229.stanford.edu/notes/cs229-notes1.pdf)

The main question is where could be the problem in my algorithm. On the other hand it would be good to know why there is a different equation for solving the problem.

Update: I removed the code snippet as it was irrelevant. I realized the algorithm works fine, what you can see on the image is just simply the lack of iterations. If you take 10000 iterations it almost perfectly fits the right result set.

$\endgroup$
1
$\begingroup$

The problem is I wasn't running enough iterations. 40 was very misleading. 1'000-10'000 is a much better number and then it fits almost perfectly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.