1
$\begingroup$

I've been led to believe that it is in fact possible to convert the least squares approximation from their partial derivatives = 0 sum form to the matrix form $\overline{b}=(M^TM)^{-1}M^T\overline{y}$. Apparently one does not get the matrix M, but combined $M^TM$ and $M^T\overline{y}$.

How might one go about the job?

For example, let the model be: $a+b\cdot \sin x$ $$S(a,b) = \sum_1^n{(y_i-a-b\cdot \sin x_i)^2}$$ $$\frac{\partial S(a,b)}{\partial a}=\sum_1^n{2(y_i-a-b\cdot \sin x_i)(-1)}$$ $$\frac{\partial S(a,b)}{\partial b}=\sum_1^n{2(y_i-a-b\cdot \sin x_i)(-\sin x_i)}$$

$$\sum y = \sum a + \sum b\sin x$$ $$\sum y\sin x = \sum a\sin x + \sum b\sin x\sin x$$

Then what?

$\endgroup$
  • $\begingroup$ Wikipedia has the derivation written out here; perhaps you could try making sense of that $\endgroup$ – Omnomnomnom Jan 31 '18 at 10:31
  • $\begingroup$ @Omnomnomnom Thanks! After some thought and comparison it began to make sense. $\endgroup$ – Felix Jan 31 '18 at 11:00
1
$\begingroup$

Based on Wikipedia provided in the comments I got it as it was actually pretty close. So by taking the constants out of the summations, one can easily bring the equation system to first a vectorized form and finally to a matrix form.

$$\sum y = a\sum 1 + b\sum \sin x$$ $$\sum y\sin x = a\sum \sin x + b\sum \sin^2 x$$ And so: $$\begin{bmatrix}\sum y \\ \sum y\sin x\end{bmatrix} = a\begin{bmatrix}\sum 1 \\ \sum \sin x\end{bmatrix} + b\begin{bmatrix}\sum\sin x \\ \sum\sin^2x\end{bmatrix}$$ $$\begin{bmatrix}\sum y \\ \sum y\sin x\end{bmatrix} = \begin{bmatrix}\sum 1 & \sum\sin x \\ \sum\sin x & \sum \sin^2 x\end{bmatrix}\begin{bmatrix}a\\b\end{bmatrix}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.