Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a rectangle given by 4 points. I'm trying to compute a transformation matrix such that the rectangle will appear straight and centered within my viewport.

I'm not even sure where to begin.

If we take the top left point to be $p0$, and work clockwise, $p1$ ... $p3$, then I think the angle of the rectangle can be computed with something like:

$$ \theta = tan^{-1}(\frac{p1.y-p0.y}{p1.x-p0.x}) $$

Which I should then be able to use to rotate around the $z$ axis using this formula:

But I'm still not sure how I get it to rotate around the center of the rectangle and then position and scale it correctly.

In C# code, here's what I've got so far:

var center = _drawingPoly.Aggregate((a, b) => a + b)/_drawingPoly.Count;

var r = _drawingPoly;
var rect = new SizeF(
    (_drawingPoly[1] - _drawingPoly[0]).Length,
    (_drawingPoly[0] - _drawingPoly[3]).Length

var cosT = (r[1].X - r[0].X) / rect.Width;
var sinT = (r[0].Y - r[1].Y) / rect.Width;

var R = new Matrix4(
    cosT, sinT, 0, 0,
    -sinT, cosT, 0, 0,
    0, 0, 1, 0,
    0, 0, 0, 1

var rectAspect = rect.Width / rect.Height;
var ctrlAspect = glControl.Width / (float)glControl.Height;

float sx, sy;

if (rectAspect < ctrlAspect)

    sy = -2f / rect.Height/1.1f;
    sx = -sy / ctrlAspect;
    sx = 2f / rect.Width/1.1f;
    sy = -sx * ctrlAspect;


float aspectRatio = glControl.Width / (float)glControl.Height;

var T = Matrix4.CreateTranslation(-center.X,-center.Y,0);
var S = Matrix4.Scale(sx, sy, 1);
var mat = T*R*S;

This mostly works, but $p0$ always has to be the top left vertex otherwise the image will get rotated. Working on a solution for that.

And here's a real life example. I'm trying to use the selection rectangle to orient this image:

share|cite|improve this question

Disclaimer: I am assuming you are working in 2D, as rectangles and viewports usually are express in 2D coordinates.

Let the four corners of the viewport be $v,v+w,v+w+h$ and $v+h$, where $v$ is the top left corner of the viewport, and $h$ and $w$ are the height and length respectively of the viewport.

Let the four corners of the rectangle be $r_0, r_1, r_2$ and $r_3$, where $r_0$ has the highest `$y$' coordinate and proceeding clockwise. The height $h_r$ and width $w_r$ of the rectangle can be calculated using Pythagoras's formula. $$ \begin{align*} h_r&=\sqrt{\left(r_3^{(x)}-r_0^{(x)}\right)^2+\left(r_3^{(y)}-r_0^{(y)}\right)^2}\\ w_r&=\sqrt{\left(r_1^{(x)}-r_0^{(x)}\right)^2+\left(r_1^{(y)}-r_0^{(y)}\right)^2} \end{align*} $$

You also need to check the aspect ratio to ensure that the rectangle isn't too wide. This will be used to determine the scale factor $s$.

if (h/w >= hr/wr) /* rectangle will use full width of the viewport with 
                       with space at the top and bottom */
    s = w/wr;
else /* space will be at the left and right */
    s = h/hr;

Now, to rotate the rectangle, scale it appropriately and align it within the viewport, a fourfold transformation is required;

  1. Translate the centre of the rectangle to the origin.
  2. Rotate the rectangle.
  3. Scale the rectangle
  4. Translate the centre of the rectangle to the centre of the viewport.

Assuming that $r_0$ will be rotated into the top left corner, the following identities are true for the rotation by an angle of $\theta$ in a counter-clockwise direction. $$ \begin{align*} \cos\theta&=\frac{r_1^{(x)}-r_0^{(x)}}{w_r}\\ \sin\theta&=\frac{r_0^{(y)}-r_1^{(y)}}{w_r} \end{align*} $$

The rotation matrix $R$ is then $$ R=\begin{bmatrix} \cos\theta&-\sin\theta\\\sin\theta&\cos\theta \end{bmatrix}. $$

The centre of the rectangle $c_r$ is able to be expressed as $$ \begin{align*} c_r^{(x)}&=\frac14\sum\limits_{i=1}^4r_i^{(x)},\\ c_r^{(y)}&=\frac14\sum\limits_{i=1}^4r_i^{(y)}, \end{align*} $$ and the centre of the viewport $c$ as $$c^{(x)}=v^{(x)}+w/2 \qquad c^{(y)}=v^{(y)}+h/2.$$

The combined transformation for each point is then $$ r_k^{new}=sR(r_k-c_r)+c, $$ where each of $r_k$, $c_r$ and $c$ are expressed as a $2\times1$ vector.

share|cite|improve this answer
"The centre of the rectangle" -- needs to be divided by 4, no? – mpen Feb 24 '13 at 23:22
@Mark Yes, that is correct. Good pickup. I will fix it. – Daryl Feb 24 '13 at 23:33
I've got the first two steps (centering and rotating), but I'm still stuck on the scaling. I was hoping to get the output as a matrix... still working on it though :-) This definitely helps. – mpen Feb 25 '13 at 1:29
@Mark $s$ should be a scalar, determined by my pseudocode halfway through the post. – Daryl Feb 25 '13 at 1:34
I got that, but I'm not just using the result to orient the 4 vertices. It's being fed into OpenGL to transform an entire image and there's still a few quirks. I'm going to have come back to this in a little bit. – mpen Feb 25 '13 at 1:38

You need to translate the viewport's origin to the center of the rectangle before doing the rotation. Then translate the center of the rectangle back to the viewport's origin.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.