I'm afraid this comes down to a long list of special cases. The calculation is best done with a simple computer program.
If we translate both squares by $(-x_1, -y_1)$ and rotate them about the origin by $-\alpha_1$ then square 1 becomes the standard unit square. I will suppose that this has already happened.
Now consider each of the four sides of square 2, with a clockwise orientation. For each side $((a,b), (c,d))$ construct the trapezium $((a,b), (c,d), (0,d), (0,b))$. Use the method below to find the area of the intersection of this trapezium with the unit square, using the convention that the area is positive if the corners are listed in clockwise order and negative otherwise. Sum these areas.
So we have reduced the problem to that of intersecting the unit square with a special kind of trapezium. This is where the case analysis starts.
If $b \geq 1$ and $d \geq 1$ then the area of intersection is zero. Similarly, if $b \leq 0$ and $d \leq 0$ then the area is zero.
If the line $y=1$ crosses the trapezium, we need to cut the trapezium into two halves along that line, and throw one of the halves away. This involves some simple algebra which I will omit here. The result is still a trapezium of the special form.
Similarly, if the line $y=0$ crosses the trapezium, we cut it in half and keep only one half.
Now it is worth testing if $a \leq 0$ and $c \leq 0$. If so, then the area is zero.
If the line $x=0$ crosses the line $((a,b), (c,d))$, we need to compute the y-coordinate of the crossing point, to cut the trapezium in half along a horizontal line through that point, and to keep one half.
Then, if the line $x=1$ crosses the line $((a,b), (c,d))$, we need to compute the y-coordinate of the crossing point, to cut the trapezium in half along a horizontal line through that point, and to keep both halves. Again this is just simple algebra and I will omit it here. Compute the area of each half separately using the method below and add the results together.
We have now reduced the problem to just two cases. Either $a \geq 1$ and $c \geq 1$ or $a \leq 1$ and $c \leq 1$. In the first case, the intersection with square 1 is just a rectangle, and its area is $b-d$. In the second case, the trapezium is entirely inside square 1, and its area is $(b-d)(a+c)/2$.