# How to do a regression with only integer values and a fixed intercept?

I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this via a CAS, but is anyone familiar with the math behind a regression of this type?

-
You might want to look at Polynomial Interpolation with your dataset and the point $(0,0)$. –  Shaktal Aug 9 '12 at 16:58
Won't this just make a "best-effort" attempt at making it pass through (0,0)? I'm okay if the residuals for the other provided data points are non-zero, but f(0) must equal 0. –  Andy Shulman Aug 9 '12 at 17:05
If you look at the first paragraph in the wikipedia article I linked to, the polynomial will go exactly through all of the points in your dataset, as $(0,0)$ will be in that dataset, $f(0)\equiv 0$. Hope this helps! N.B: This method will only deal with no duplicate $x$ values with differing $y$ values, I guess if you have duplicate $x$ values with different $y$ values, you could take an average of the $y$ values and just use the point $(x,y_{\text{avg}})$ in your dataset. –  Shaktal Aug 9 '12 at 17:10

As always with least-squares regression, we look to the definitions for starters. I'll do the case of fitting a straight line through the origin; the case of higher-degree polynomials will involve partial derivatives, but is otherwise straightforward.

We want to fit the model $y=mx$ to the data points $(x_k,y_k)$ for $k=1,\dots,n$. Least squares regression demands the finding the value of $m$ that minimizes the sum of squares of the residuals:

$$S=\frac12\sum_{k=1}^n (m x_k-y_k)^2$$

To do that, we differentiate with respect to $m$, and equate to zero:

$$\frac{\mathrm dS}{\mathrm dm}=\sum_{k=1}^n (m x_k-y_k)x_k=0$$

and it is rather easy to solve for $m$:

\begin{align*} \sum_{k=1}^n (m x_k-y_k)x_k&=0\\ m \sum_{k=1}^n x_k^2-\sum_{k=1}^n x_k y_k&=0\\ \end{align*}

and we end up with

$$\color{blue}{m=\frac{\sum\limits_{k=1}^n x_k y_k}{\sum\limits_{k=1}^n x_k^2}}$$

In words: multiply the abscissas and ordinates together, total them up, and divide this total by the sum of the squares of the abscissas.

If one wants to regress instead with respect to some arbitrary fixed point $(h,k)$, subtract out $h$ from the abscissas and $k$ from the ordinates, perform the procedure above, and then undo the translation accordingly.

-
Thanks! Two questions: 1) How would I go about this for a polynomial of degree $n>1$. 2) If my fixed point is $(0,0)$, then it seems like I wouldn't do any adjustments, but this doesn't make sense to me. If I gave you a set of points perfectly describing the curve $y=x+1$ but I wanted the regression to pass through $(0,0)$, not adjusting the regression clearly does not work. –  Andy Shulman Aug 10 '12 at 0:44
Andy, as I said, "the case of higher-degree polynomials will involve partial derivatives". Thus, if for example you're fitting with the model $y=ax^2+bx$, you now have to take partial derivatives of the objective function $$\frac12\sum_k (ax_k^2+bx_k-y_k)^2$$ with respect to $a$ and $b$ (that is, the gradient), equate both partial derivatives to zero, and solve the resulting equations for $a$ and $b$, –  Ｊ. Ｍ. Aug 10 '12 at 0:48
@Andy, "but I wanted the regression to pass through $(0,0)$" - if you fit with the model $y=mx$, then the fitting function is guaranteed to pass through the origin. Try it yourself. –  Ｊ. Ｍ. Aug 10 '12 at 0:49
Hah. Yes, I suppose it would have to. I mentally added the $+b$ part. Great, thanks for your help. –  Andy Shulman Aug 10 '12 at 13:03