Currently I am learning the Linear Regression, in particular, the cost function. Here is the problem I am working on right now:
Suppose we have a training set with $m=3$ examples-points $(1,1), (2,2)$ and $(3,3)$.The hypothesis function is $h_\theta(x)=\theta_1x$ with a parameter $\theta_1$. The cost function is $J(\theta_1)=\frac{1}{2m}\sum_{i=0}^m(h_\theta(x^i)-y^i)^2$ . We need to find $J(0)$ , which is a relatively easy task if done manually(and I have already done it).
I am interested in doing it through a derivative. If I do it this way, I get $J'(\theta_1)=\frac{1}{2m}\sum_{i=1}^m(2(h_{\theta_1}x^i-y^i))h_\theta$ (If hopefully I haven't done any mistakes) Then, to find a minimum value(values) of $\theta_1$ all I need to do is to solve $J'(\theta_1)=0$. That's where I have a few questions.
Can we assume that the sum will never be zero? If so, when I solve this equation I find that the only way for the equation to be zero is for $h_{\theta_1}$ to be zero, which doesn't seem right, or for $2(h_{\theta_1}x^i-y^i)$ to be zero for any pair of $(x,y)$. That is, $\theta_1=1$ . Is my reasoning correct?