0
$\begingroup$

I have to fit a 3rd degree polynomial on a small and dense dataset. I've used polynomial regression and least square error, but the result is undesirable. I will explain why.

Let the polynomial be: $$y = c_0+c_1x+\frac{1}{2}c_2x^2+\frac{1}{6}c_3x^3$$

And the dataset: $$x = [27, 32, 33, 34, 35, 38, 39, 40, 41, 42, 43]$$ $$y = [29, 29, 29, 29, 29, 29, 30, 30, 29, 29, 29]$$

By applying polynomial regression, I get

I understand that this is absolutely correct. It fits on my data. But the desired behavior on this kind of points is as follows (made by hand)

Also, if the points describe a curvature, the resulting polynomial should follow that curvature.

What I can't do to get to this result

  • Modify input dataset
  • Modify ecuation

What I can do

  • Modify anything about polynomial regression (like error function)
  • Replace polynomial regression

I've also tried Levenberg-Marquardt, but as expected the results are the same as polynomial regression.

$\endgroup$
  • $\begingroup$ You want to have a linear look on your solution, but you use a cubic? Why not simply estimate a linear function? $\endgroup$ – Johan Löfberg Aug 24 '17 at 6:55
  • $\begingroup$ @JohanLöfberg, sometimes I will receive points that can be described the best using this function. I don't know when I will receive this kind of points. That's why I'm forced to use it. $\endgroup$ – Shockwave Aug 24 '17 at 7:47
  • $\begingroup$ Then I would use $\ell_1$ as it appears more robust for your purpose. $\endgroup$ – Johan Löfberg Aug 24 '17 at 8:19
0
$\begingroup$

For sure, the cubic fits your data as shown below $$\left( \begin{array}{ccc} x & y & \text{calc}\\ 27 & 29 & 29.030 \\ 32 & 29 & 28.832 \\ 33 & 29 & 28.938 \\ 34 & 29 & 29.063 \\ 35 & 29 & 29.193 \\ 38 & 29 & 29.490 \\ 39 & 30 & 29.514 \\ 40 & 30 & 29.480 \\ 41 & 29 & 29.375 \\ 42 & 29 & 29.186 \\ 43 & 29 & 28.899 \end{array} \right)$$ The problem is that, for the parameters, one can obtain $$\begin{array}{clclclclc} \text{} & \text{Estimate} & \text{Standard Error} & \text{Confidence Interval} \\ c_0 & 112.129 & 52.7599 & \{-16.9699,241.228\} \\ c_1 & -7.49163 & 4.64423 & \{-18.8557,3.87239\} \\ c_2 & 0.442603 & 0.269298 & \{-0.216346,1.10155\} \\ c_3 & -0.0128499 & 0.00771852 & \{-0.0317365,0.00603661\} \\ \end{array}$$ and none of them is significant.

$\endgroup$
0
$\begingroup$

Besides the trivial solution to simply use an affine model instead of a cubic, if you switch from a quadratic residual to absolute value, you will obtain a affine response.

With absolute values, called $\ell_1$, the solution is typically less sensitive to outliers, and in your case the two values of $30$ will effectively be seen as outliers and the $\ell_1$-optimal solution is $y(x) = 29$.

The $\ell_1$ model can be cast as a linear program by noting that minimizing $\sum |e_i|$ can be written as minimize $\sum t_i$ subject to $-t_i \leq e_i \leq t_i$.

$\endgroup$
  • $\begingroup$ Interesting. After a few examples over the internet it may be promising. I will try to implement this and I will come back with feedback. Thank you. $\endgroup$ – Shockwave Aug 24 '17 at 12:16
  • $\begingroup$ Note that I fixed a typo in the LP formulation (removed absolute value) $\endgroup$ – Johan Löfberg Aug 24 '17 at 12:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.