I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this via a CAS, but is anyone familiar with the math behind a regression of this type?
$\begingroup$
$\endgroup$
3
-
$\begingroup$ You might want to look at Polynomial Interpolation with your dataset and the point $(0,0)$. $\endgroup$– Thomas RussellAug 9, 2012 at 16:58
-
$\begingroup$ Won't this just make a "best-effort" attempt at making it pass through (0,0)? I'm okay if the residuals for the other provided data points are non-zero, but f(0) must equal 0. $\endgroup$– Andy ShulmanAug 9, 2012 at 17:05
-
$\begingroup$ If you look at the first paragraph in the wikipedia article I linked to, the polynomial will go exactly through all of the points in your dataset, as $(0,0)$ will be in that dataset, $f(0)\equiv 0$. Hope this helps! N.B: This method will only deal with no duplicate $x$ values with differing $y$ values, I guess if you have duplicate $x$ values with different $y$ values, you could take an average of the $y$ values and just use the point $(x,y_{\text{avg}})$ in your dataset. $\endgroup$– Thomas RussellAug 9, 2012 at 17:10
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
As always with least-squares regression, we look to the definitions for starters. I'll do the case of fitting a straight line through the origin; the case of higher-degree polynomials will involve partial derivatives, but is otherwise straightforward.
We want to fit the model $y=mx$ to the data points $(x_k,y_k)$ for $k=1,\dots,n$. Least squares regression demands the finding the value of $m$ that minimizes the sum of squares of the residuals:
$$S=\frac12\sum_{k=1}^n (m x_k-y_k)^2$$
To do that, we differentiate with respect to $m$, and equate to zero:
$$\frac{\mathrm dS}{\mathrm dm}=\sum_{k=1}^n (m x_k-y_k)x_k=0$$
and it is rather easy to solve for $m$:
$$\begin{align*} \sum_{k=1}^n (m x_k-y_k)x_k&=0\\ m \sum_{k=1}^n x_k^2-\sum_{k=1}^n x_k y_k&=0\\ \end{align*}$$
and we end up with
$$\color{blue}{m=\frac{\sum\limits_{k=1}^n x_k y_k}{\sum\limits_{k=1}^n x_k^2}}$$
In words: multiply the abscissas and ordinates together, total them up, and divide this total by the sum of the squares of the abscissas.
If one wants to regress instead with respect to some arbitrary fixed point $(h,k)$, subtract out $h$ from the abscissas and $k$ from the ordinates, perform the procedure above, and then undo the translation accordingly.
answered Aug 9, 2012 at 17:27
J. M. ain't a mathematicianJ. M. ain't a mathematician
74.5k8 gold badges208 silver badges339 bronze badges
-
$\begingroup$ Thanks! Two questions: 1) How would I go about this for a polynomial of degree $n>1$. 2) If my fixed point is $(0,0)$, then it seems like I wouldn't do any adjustments, but this doesn't make sense to me. If I gave you a set of points perfectly describing the curve $y=x+1$ but I wanted the regression to pass through $(0,0)$, not adjusting the regression clearly does not work. $\endgroup$ Aug 10, 2012 at 0:44
-
$\begingroup$ Andy, as I said, "the case of higher-degree polynomials will involve partial derivatives". Thus, if for example you're fitting with the model $y=ax^2+bx$, you now have to take partial derivatives of the objective function $$\frac12\sum_k (ax_k^2+bx_k-y_k)^2$$ with respect to $a$ and $b$ (that is, the gradient), equate both partial derivatives to zero, and solve the resulting equations for $a$ and $b$, $\endgroup$ Aug 10, 2012 at 0:48
-
$\begingroup$ @Andy, "but I wanted the regression to pass through $(0,0)$" - if you fit with the model $y=mx$, then the fitting function is guaranteed to pass through the origin. Try it yourself. $\endgroup$ Aug 10, 2012 at 0:49
-
$\begingroup$ Hah. Yes, I suppose it would have to. I mentally added the $+b$ part. Great, thanks for your help. $\endgroup$ Aug 10, 2012 at 13:03
$\begingroup$
$\endgroup$
If the response variable is a count there are special regression models for that including Poisson regression and negative binomial regression. You might want to look at the following books involving models for count data:
Regression Analysis of Count Data; A. Colin Cameron, Pravin K. Trivedi
Negative Binomial Regression; Joseph M. Hilbe
Generalized Linear Models and Extensions, Third Edition; James W. Hardin, Joseph M. Hilbe
J. M. ain't a mathematician
74.5k8 gold badges208 silver badges339 bronze badges
answered Aug 9, 2012 at 18:04