# Line Of Best Fit With Perpendicular Error

The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?

• Make a higher dimensional level set instead. – mathreadler Dec 21 '18 at 22:01

This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:

Note : For the choice of sign, compute $$\sum_{k=1}^n(ax_k+b-y_k)^2$$ in both cases and keep the smaller.

Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis

A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.

Numerical example with the principal component method :

Comparison with the above least mean square offset method :

The results of both methods are exactly the same, as it is analytically expected.

Graphical representation :

• Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ? – Claude Leibovici Dec 22 '18 at 6:31
• Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable. – JJacquelin Dec 22 '18 at 7:30
• Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before. – Claude Leibovici Dec 22 '18 at 7:46
• @Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas. – JJacquelin Dec 22 '18 at 8:53
• Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-) – Claude Leibovici Dec 22 '18 at 8:56

Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.

So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$ We fit to: $$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$

This will automatically handle all your roubles ( if you add some tea at the start ).