I'm having trouble seeing the relationship in the following equation. Let's assume $J(0,1)$ and $m=4$.

First I figure out my hypothesis function $$\begin{array}{rcl}h_\Theta(x)&=&\Theta_0+\Theta_1x\\&=&0+1x\\&=&x\end{array}$$

So my cost function should look like

$$J(\Theta_0,\Theta_1)=\frac{1}{2\times 4}\sum_{i=1}^m\left[h_\Theta(x^{(i)})-y^{(i)}\right]^2$$

This is where I become confused. Since our $h_\Theta(x)$ is $x$, isn't our $h_\Theta(x^{(i)})$ just $x^{(i)}$? So why isn't it: $$\begin{array}{rcl}J(\Theta_0,\Theta_1)&=&\frac{1}{8}[(1-1)^2+(2-2)^2+(3-3)^2+(4-4)^2]\\&=&\frac{1}{8}[0^2+0^2+0^2+0^2]\\&=&0\end{array}$$

I know the right way to do it is: $$\begin{array}{rcl}J(\Theta_0,\Theta_1)&=&\frac{1}{8}(1^2+1^2+1^2+1^2)\\&=&\frac{1}{8}\times 4\;=\;\frac{4}{8}\\&=&0.5\end{array}$$

Could someone explain to me how we get $1^2+1^2+1^2+1^2$.



I think You are confusing the notation. The super-index $n^{(i)}$ denotes the $i$-th element of your training set. For instance, you have the experimental data of $m$ pairs $$\begin{array}{l|lllll}x^{(i)}&x^{(1)}&x^{(2)}&\;\dotso\;&x^{(m)}\\\hline y^{(i)}&y^{(1)}&y^{(2)}&\;\dotso\;&y^{(m)}\end{array}$$ The hypothesis function does not compute the ouput $y^{(i)}$, but it computes the "approximated" value to $y^{(i)}$.

You have $(1−1)^2+(2−2)^2+(3−3)^2+(4−4)^2$ if $x^{(i)}=i$ and $y^{(i)}=i$. But this is a trivial training set. By other hand, you obtain $1^2+1^2+1^2+1^2$ if you data looks like $$\begin{array}{l|lllll}x^{(i)}&a&a&\;\dotso\;&a\\\hline y^{(i)}&a+1&a+1&\;\dotso\;&a+1\end{array}$$ or $$\begin{array}{l|lllll}x^{(i)}&a+1&a+1&\;\dotso\;&a+1\\\hline y^{(i)}&a&a&\;\dotso\;&a\end{array}$$ for some value $a$.

Note that this is true if your hypothesis function is still $h_\Theta(x) = x$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.