3
$\begingroup$

Let us have five points at $x = -2,-1,0,1,2$ with ordinates equal to $y_i$, I want to derive the formula for $a_0$, such that the parabola $y = a_0 + a_1x + a_2x^2$ fits the points the best in ordinary least square sense.

$$(y_{-2}-a_0 +2a_1-4a_2)^2 + (y_{-1} - a_0 + a_1 -a_2)^2 + (y_{0} -a_0)^2 + (y_{1} -a_0 - a_1 - a_2)^2 + (y_{2} - a_0 - 2a_2 - 4a_2)^2 \to min \ \text{w.r.t. } a_0, a_1 \text{and } a_2.$$

I know the answer $a_0 = \frac{1}{35} [ -3 y_{-2} + 12 y_{-1} + 17 y_0 + 12 y_1 -3 y_2]$. It was given on a lecture and the proof was left as an exercise.

Background of the question is time series weighted moving average smoothing. We substitute every $y$ with $a_0$ to get a smoother time series.

However I don't understand how we can derive it not taking three derivatives, getting 3 linear equations and solving them. But looks like there is a simpler proof for $a_0$ particularly that I don't see. If we denote every summand being squared as $S_i$ then we get a system of linear equations.

$$ \begin{cases} S_{-2} + S_{-1} + S_0 + S_1 + S_2 = 0 \\ 2S_{-2}+S_{-1} - S_1 - 2S_2 = 0 \\ 4S_{-2}+S_{-1}+S_{1}+4S_2 = 0 \end{cases}$$

But is there a faster proof? The original slide is given below for reference

weighted moving average

$\endgroup$
1
$\begingroup$

I don't know about faster, but I managed to simplify the task by exploiting a few symmetries. Very likely I am missing something major, for this is not really my cup of coffee.

We are looking for a weighted average, so it is clear that the solution must have the form $$ \hat{a}_0=L(y_{-2},\ldots,y_2)=\sum_{i=-2}^2q_iy_i, $$ where the constants $q_i$ are the weights. Clearly we must have $$\sum_{i=-2}^2q_i=1\tag{1}$$ for this to be an average.

  • The weighted average is immune to a change of direction of time. In other words, for all inputs $(y_i)_{-2\le i\le 2}$, we must have $$L(y_2,y_1,y_0,y_{-1},y_{-2})=L(y_{-2},y_{-1},y_0,y_1,y_2).$$ For this to be possible we must have the symmetries $q_{-1}=q_1$ as well as $q_{-2}=q_2$.
  • If we add a quadratic term to the input, this will not affect the value of $\hat{a}_0$, for such a change will be absorbed by $\hat{a}_2$. In other words, we must have $$L(4,1,0,1,4)=0$$ as that input matches perfectly with $y_i=i^2$. Together with the symmetries this gives us the equation $$q_1+4q_2=0.\tag{2}$$

So far so good. We have three unknowns, $q_0,q_1,q_2$ and two equations, $(1)$ and $(2)$. To solve for them we need something extra. Here I'm sad to say I had to brute force it. Consider a pulse input $(y_i)_{-2\le i\le 2}=(0,0,1,0,0)$. That is $y_0=1$ and the rest all zero. Again by symmetry, the best quadratic approximation has no linear term, so we test this against $y_i=a_0+a_2i^2$. The optimal $a_0$ will then be $L(0,0,1,0,0)=q_0$ allowing us to solve the problem.

The sum of squared errors will then be $$ E(a_0,a_2)=(1-a_0)^2+2(a_2+a_0)^2+2(4a_2+a_0)^2=1-2a_0+5a_0^2+20a_0a_2+34a_2^2. $$ Solving the system $\partial E/\partial a_0=0, \partial E/\partial a_2=0$ is easy, and yields $$a_0=17/35, a_2=-1/7.\tag{3}$$ As the only critical point that must yield the optimum.

So we can conclude that $q_0=17/35$, whence $(1)$ together with the symmetries gives $$q_1+q_2=9/35\tag{4}.$$ Combining this with $(2)$ leads to the given solution.

$\endgroup$
  • $\begingroup$ Knowing from elsewhere on the site that the OP is also studying algebra I could not resist checking how far symmetries alone will take us. Unfortunately didn't quite take us the distance, but I was too far gone to stop :-( Anyway, the estimate is linear, so it is natural to consider a single pulse as the input. $\endgroup$ – Jyrki Lahtonen Sep 1 at 18:01
  • $\begingroup$ Updated my profile info, I appreciate your trait, giving the answer considering the asker's background. $\endgroup$ – Lada Dudnikova Sep 3 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.