Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this via a CAS, but is anyone familiar with the math behind a regression of this type?

share|improve this question
    
You might want to look at Polynomial Interpolation with your dataset and the point $(0,0)$. –  Shaktal Aug 9 '12 at 16:58
    
Won't this just make a "best-effort" attempt at making it pass through (0,0)? I'm okay if the residuals for the other provided data points are non-zero, but f(0) must equal 0. –  Andy Shulman Aug 9 '12 at 17:05
    
If you look at the first paragraph in the wikipedia article I linked to, the polynomial will go exactly through all of the points in your dataset, as $(0,0)$ will be in that dataset, $f(0)\equiv 0$. Hope this helps! N.B: This method will only deal with no duplicate $x$ values with differing $y$ values, I guess if you have duplicate $x$ values with different $y$ values, you could take an average of the $y$ values and just use the point $(x,y_{\text{avg}})$ in your dataset. –  Shaktal Aug 9 '12 at 17:10
add comment

2 Answers

up vote 1 down vote accepted

As always with least-squares regression, we look to the definitions for starters. I'll do the case of fitting a straight line through the origin; the case of higher-degree polynomials will involve partial derivatives, but is otherwise straightforward.

We want to fit the model $y=mx$ to the data points $(x_k,y_k)$ for $k=1,\dots,n$. Least squares regression demands the finding the value of $m$ that minimizes the sum of squares of the residuals:

$$S=\frac12\sum_{k=1}^n (m x_k-y_k)^2$$

To do that, we differentiate with respect to $m$, and equate to zero:

$$\frac{\mathrm dS}{\mathrm dm}=\sum_{k=1}^n (m x_k-y_k)x_k=0$$

and it is rather easy to solve for $m$:

$$\begin{align*} \sum_{k=1}^n (m x_k-y_k)x_k&=0\\ m \sum_{k=1}^n x_k^2-\sum_{k=1}^n x_k y_k&=0\\ \end{align*}$$

and we end up with

$$\color{blue}{m=\frac{\sum\limits_{k=1}^n x_k y_k}{\sum\limits_{k=1}^n x_k^2}}$$

In words: multiply the abscissas and ordinates together, total them up, and divide this total by the sum of the squares of the abscissas.

If one wants to regress instead with respect to some arbitrary fixed point $(h,k)$, subtract out $h$ from the abscissas and $k$ from the ordinates, perform the procedure above, and then undo the translation accordingly.

share|improve this answer
    
Thanks! Two questions: 1) How would I go about this for a polynomial of degree $n>1$. 2) If my fixed point is $(0,0)$, then it seems like I wouldn't do any adjustments, but this doesn't make sense to me. If I gave you a set of points perfectly describing the curve $y=x+1$ but I wanted the regression to pass through $(0,0)$, not adjusting the regression clearly does not work. –  Andy Shulman Aug 10 '12 at 0:44
    
Andy, as I said, "the case of higher-degree polynomials will involve partial derivatives". Thus, if for example you're fitting with the model $y=ax^2+bx$, you now have to take partial derivatives of the objective function $$\frac12\sum_k (ax_k^2+bx_k-y_k)^2$$ with respect to $a$ and $b$ (that is, the gradient), equate both partial derivatives to zero, and solve the resulting equations for $a$ and $b$, –  J. M. Aug 10 '12 at 0:48
    
@Andy, "but I wanted the regression to pass through $(0,0)$" - if you fit with the model $y=mx$, then the fitting function is guaranteed to pass through the origin. Try it yourself. –  J. M. Aug 10 '12 at 0:49
    
Hah. Yes, I suppose it would have to. I mentally added the $+b$ part. Great, thanks for your help. –  Andy Shulman Aug 10 '12 at 13:03
add comment

If the response variable is a count there are special regression models for that including Poisson regression and negative binomial regression. You might want to look at the following books involving models for count data:

Regression Analysis of Count Data; A. Colin Cameron, Pravin K. Trivedi

Negative Binomial Regression; Joseph M. Hilbe

Generalized Linear Models and Extensions, Third Edition; James W. Hardin, Joseph M. Hilbe

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.