0
$\begingroup$

Let $$F(a,b) = \sum_{i=1}^n \left[ y_i - (ax_i+b) \right]^2$$

Find the minimum of $F$.

Evaluating the dirctional derivatives:

$$\frac{dF}{da} = \sum_{n=1}^n 2\cdot (y_i - (ax_i+b))(-x_i) \\ \frac{dF}{db} = \sum_{n=1}^n 2\cdot (y_i - (ax_i+b))$$

Now, we want $\nabla F = (\frac{dF}{da}, \frac{dF}{db}) = (0,0)$.

So we have (after some algebra):

$$ \sum_{n=1}^n y_i = \sum_{n=1}^n (ax_i+b)(-x_i) \\ \sum_{n=1}^n y_i = \sum_{n=1}^n ax_i+b$$

So far so good? How should I continue? (this two equations relatively complicated)

$\endgroup$
2
  • $\begingroup$ why not plugging the first equality in the second one, or the other way round? $\endgroup$
    – tired
    Jan 19, 2015 at 15:25
  • $\begingroup$ Anyway I just used your equations. The first one is not fine. I would correct my answer $\endgroup$
    – rlartiga
    Jan 19, 2015 at 15:37

1 Answer 1

2
$\begingroup$

The first condition is: $$\sum_{n=1}^n 2\cdot (y_i - (ax_i+b))(-x_i) =0 \iff \sum_{n=1}^n 2\cdot (-y_ix_i + (ax_i+b)(x_i)) =0$$ $$ \iff \sum_{n=1}^n y_ix_i = \sum_{n=1}^n (ax_i+b)(x_i) \iff \sum_{n=1}^n y_ix_i =a \sum_{n=1}^n x_i^2 +b \sum_{n=1}^n x_i$$ $$\iff \overline{xy}=\overline{x^2}a+b\bar{x}$$

The second one is: $$\sum_{n=1}^n y_i = \sum_{n=1}^n ax_i+b \iff \bar{y}=a\bar{x}+b $$

Then you have two equations:

$$\overline{xy}=\overline{x^2}a+b\bar{x}$$ $$\bar{y}=a\bar{x}+b $$

Which gives as solution:

$$\hat a=\frac{\overline{xy}-\bar{x}\bar{y}}{\overline{x^2}-\bar{x}^2}$$ $$\hat b=\bar{y}-\hat a \bar{x}$$

Which is the same solution you can find in any text of statistics or wiki

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .