Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Qt has a class QRect which tells whether the point is inside the rectangle or not.

Now, the problem is to find out on which outer side (out of four) of the rectangle does the point lie.

The equation that I already have is:

C++ code:

double m              = (y2 - y1) / (x2 - x1);
double equationResult = (newY - y1) - ((m * newX) + (m * x1));

if (equationResult < 0)
      return "in";
      return "out";

where newX and newY are the points which we are supposed to check.

The point lies on the right hand side of the rectangle.

So, with this assumption in mind when I supplied the end points of the top of the rectangle, the result I got was "out".
Even with the end points of the right side, the result I got was "out".

So, if more than one sides are going to give the same output, how am I supposed to know the position of the point?

share|cite|improve this question
If all of those variables are double, you might just fall victim to some rounding errors. – user22805 Sep 12 '12 at 5:56
@DavidWallace so, what should be their type? – TheIndependentAquarius Sep 12 '12 at 5:58
I feel that really depends on your problem domain. Is this some kind of graphics program that you're writing? – user22805 Sep 12 '12 at 6:03
There are closely related questions at… and… – user22805 Sep 12 '12 at 6:11
@DavidWallace That's why I mentioned in the start of this pots that Qt already has a function for determining whether the point in inside or not, so that is not the problem. – TheIndependentAquarius Sep 12 '12 at 6:13
up vote 1 down vote accepted

Why not : $\ \mathrm{in}:=(x_1\le x \le x_2) \land (y_1\le y \le y_2)\ $ ?

(supposing that $x_1\le x_2$ and $y_1\le y_2$ : i.e. the rectangle is first 'normalized')

Let's illustrate the 9 regions (larger values of $y$ are at the bottom) :


But (from David's discussion +1) perhaps that you are rather interested by these four regions :

enter image description here

with $\ x_c=\dfrac {x_1+x_2}2,\ y_c=\dfrac {y_1+y_2}2$

share|cite|improve this answer
what is the symbol /\ between the brackets? – TheIndependentAquarius Sep 12 '12 at 6:50
@AnishaKaul: 'logical and' – Raymond Manzoni Sep 12 '12 at 6:51
Ah, you mean &. Is my equation faulty? – TheIndependentAquarius Sep 12 '12 at 6:54
@AnishaKaul: in fact && and yes (one test without || is not enough). The problem is that there are 9 regions in all (from your point of view). My first test limits to the 3 central vertical regions, the second to the 3 central horizontal regions. – Raymond Manzoni Sep 12 '12 at 6:57
@AnishaKaul: (I updated my answer) I think that you wanted to test : $\dfrac {y-y_1}{y_2-y_1}<\dfrac {x-x_1}{x_2-x_1}$ (supposing that $\cdots-(m*x_1)$ : you had too a sign problem...). But this is true for half the plane, this is why I have four tests (we could reduce that to two using absolute value). – Raymond Manzoni Sep 12 '12 at 7:46

I think there's an error in your code. It seems to me that this will return "in" for points that are below the line, and "out" for points that are on or above the line. So there is a region outside the rectangle, where you'll actually get "in" for all four lines. This is aside from the issue of rounding errors that I mentioned in my comment.


I think from our conversation in the chat room that you actually want to know which of the four main directions you should move the centre of your square, to most closely approach the target point. So you want to know whether the X difference or the Y difference is greater.

Write xDifference = xTarget - xCentre and yDifference = yTarget - yCentre. Then, there are basically four cases.

  • If xDifference > 0 && xDifference > abs( yDifference ), you want to go to the right.
  • If xDifference <= 0 && xDifference < - abs( yDifference ), you want to go to the left.
  • If yDifference > 0 && yDifference >= abs( xDifference ), you want to go upwards.
  • If yDifference <= 0 && yDifference <= - abs( xDifference ), you want to go downwards.

Note that the differences between <= and < aren't too significant here; likewise the differences between >= and >. Basically if you need to head exactly north-east, or south-west or whatever to get to your target point, you've got a choice of two directions to head in first.

Please forgive the mixture of C++ notation and mathematical notation that I've used here - I've done this deliberately in the hope that the OP will understand this better.

share|cite|improve this answer
I understood that $(x_y,y_1)$ and $(x_2,y_2)$ are endpoints of one of the four sides and she performs correponding tests for all four sides. (Though I too first thought they were lower left and top right corner of the rect) – Hagen von Eitzen Sep 12 '12 at 6:06
Yes, I understand it the same way that you do. Anisha appears to be applying this test to one side at a time. This will only be possible if the rectangle is NOT aligned parallel to the axes, otherwise two of the sides won't fit this form. – user22805 Sep 12 '12 at 6:08
Well, if you think there's an error, then please show the correct equation by correcting mine. BTW, this is not any graphics thing at all. I have a map tile, and I need to find the direction of a latitude-longitude point. – TheIndependentAquarius Sep 12 '12 at 6:10

If the point is "out" by checking the lower edge and is out by checking the left edge, then it is obviously posiotioned to the lower-left of the rectangle. Extending the four sided of the rectangle splits the plane into 9 domains, four of which correspond to only one "out" test and four corresponding to two "out" tests.

share|cite|improve this answer
No, the point is "out" when I check top side. How am I su[pposed to correct this equation? – TheIndependentAquarius Sep 12 '12 at 6:12

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.