Rotating regular polygon Let $R_1$ be regular $n$-sided polygon on the plane (square, pentagon, hexagon, etc).
Now from this position we start to rotate this polygon about its center of gravity obtaining figure $R_2$. 


*

*How to calculate the angle of rotation $\alpha$ for the case where common area
of  $R_1$ and $R_2$  i.e. area ($R_1 \cap R_2)$  will be minimal? (intuition tells what possible solution could be but how to prove it?)

*Does some simple method exist  for solution of this problem in general case? (preferably with the use of rotations matrices)

*The procedure for  $n$-odd and $n$-even could be  the same or we should to differentiate between these two cases?


Additionally:


*

*Could it be proven that the shape obtained for situation of minimal
area is also a regular polygon ( $2n$ sided) as we see in the below picture of pentagon made by Joseph?

 A: The minimum is achieved at $\alpha = \frac{\pi}{n}$ and at minimum, $R_1 \cap R_2$ is a regular $2n$-gon.
Choose a coordinate system so that $R_1$ is centered at origin
and one of its vertices lies on $x$-axis.
Let $\rho(\theta)$ be the function which allow us to parametrize $\partial R_1$ in following manner:
$$\mathbb{R} \ni \theta \quad\mapsto\quad (x,y) = (\sqrt{2\rho(\theta)}\cos\theta,\sqrt{2\rho(\theta)}\sin\theta) \in \partial R_1$$
In terms of $\rho(\theta)$, we have
$$f(\alpha) \stackrel{def}{=} \verb/Area/(R_1 \cap R_2) = \int_0^{2\pi} \min(\rho(\theta),\rho(\theta-\alpha)) d\theta$$
Since $R_1$ is a regular $n$-gon and one of its vertices lies on $x$-axis, $\rho(\theta)$ is even and periodic with period $\frac{2\pi}{n}$. In fact, it 
strictly decreases on $[0,\frac{\pi}{n}]$ and strictly increases on $[\frac{\pi}{n},\frac{2\pi}{n}]$.
As a result of these, $f(\alpha)$ is even and periodic with same period. To determine the minimum of $f(\alpha)$, we only need to study the case
where $\alpha \in \left[0,\frac{\pi}{n}\right]$.
For $\alpha \in \left[0,\frac{\pi}{n}\right]$ and $\theta \in \left[0,\frac{2\pi}{n}\right]$, the curve $\rho(\theta)$ and $\rho(\theta - \alpha)$ intersect at
$\frac{\alpha}{2}$ and $\frac{\alpha}{2} + \frac{\pi}{n}$.
This leads to
$$\begin{align}f(\alpha) 
&= n\left[
\int_{\frac{\alpha}{2}}^{\frac{\alpha}{2}+\frac{\pi}{n}} \rho(\theta) d\theta 
+ \left(
\int_0^{\frac{\alpha}{2}} + \int_{\frac{\alpha}{2}+\frac{\pi}{n}}^{\frac{2\pi}{n}}
\right)\rho(\theta-\alpha)d\theta
\right]
= 2n\int_{\frac{\alpha}{2}}^{\frac{\alpha}{2}+\frac{\pi}{n}} \rho(\theta) d\theta\\
\implies
\frac{df(\alpha)}{d\alpha} &= n\left(\rho\left(\frac{\alpha}{2}+\frac{\pi}{n}\right) - \rho\left(\frac{\alpha}{2}\right)\right)
\end{align} 
$$
At the minimum, we have 
$$\frac{df(\alpha)}{d\alpha} = 0
\implies
\rho\left(\frac{\alpha}{2}\right) = \rho\left(\frac{\alpha}{2} + \frac{\pi}{n}\right)
= \rho\left(\frac{\alpha}{2} - \frac{\pi}{n}\right) = \rho\left(\frac{\pi}{n} - \frac{\alpha}{2}\right)
$$
But $\frac{\pi}{n} - \frac{\alpha}{2}$ also belongs to $[0,\frac{\pi}{n}]$ and $\rho(\theta)$ is strictly decreasing there, this means
$$\frac{\alpha}{2} = \frac{\pi}{n} - \frac{\alpha}{2}\quad\implies\quad \alpha = \frac{\pi}{n}$$
Please note that this argument doesn't use the explicit form of regular $n$-gon.
It uses 


*

*$n$-fold rotation symmetry about center,

*$2$-fold reflection symmetry about a ray through a vertex,

*$\rho(\theta)$ is strictly decreasing on suitable intervals of $\theta$. 


This means the same argument should work for other shapes with similar properties.
e.g. those obtain from filling the "interior" of a regular star polygon.
A: 
"Does some simple method exist for solution of this problem in general case?"

Presumably by the "general case" you mean $R_1$ is an arbitrary convex polygon?
Or maybe an arbitrary simple polygon, perhaps nonconvex?
I don't think this will have a simple answer.
Below I computed that the minimum intersection area for the blue 
quadrilateral is achieved with
$R_2$ rotated about $94.5^\circ$ degrees.
In general you might have to resort to numerical optimization.

           


And, Yes, the minimum for a regular pentagon is achieved at a rotation of
$36^\circ$, half the $72^\circ$ angle subtended by each edge from the centroid. Etc.

           


