The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?

  • $\begingroup$ Make a higher dimensional level set instead. $\endgroup$ – mathreadler Dec 21 '18 at 22:01

This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:

enter image description here

Note : For the choice of sign, compute $\sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.

Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html

Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis

A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.


Numerical example with the principal component method :

enter image description here

Comparison with the above least mean square offset method :

enter image description here

The results of both methods are exactly the same, as it is analytically expected.

Graphical representation :

enter image description here

  • $\begingroup$ Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ? $\endgroup$ – Claude Leibovici Dec 22 '18 at 6:31
  • $\begingroup$ Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable. $\endgroup$ – JJacquelin Dec 22 '18 at 7:30
  • $\begingroup$ Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before. $\endgroup$ – Claude Leibovici Dec 22 '18 at 7:46
  • $\begingroup$ @Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas. $\endgroup$ – JJacquelin Dec 22 '18 at 8:53
  • $\begingroup$ Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-) $\endgroup$ – Claude Leibovici Dec 22 '18 at 8:56

Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.

So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$ We fit to: $$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$

This will automatically handle all your roubles ( if you add some tea at the start ).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.