Finding the Center of Mass of a disk when a part of it is cut out. From a uniform disk of radius $R$ a circular disk of radius $\frac{R}{2}$ is being cut out.

The center of the "cut out" disk is at $R/2$ from the venter of the original disk. We have to find the center of mass of leftover body.
I thought that we should set up a coordinate system with the center of original disk as the origin. The formula for center of mass is
$$
\vec{R}_{CM} = \frac{1}{M_{tot}} \int \vec{r} ~dm$$
I thought of creating another identical region (identical to what is being cut out) on the left of $O$. Like this

By symmetry, any position vector $\vec{r}$ outside the encircled region (on the left) will have it's counter-part and hence it will cancel up. So, we need to worry only about the integral inside the circular region, even there by symmetry we know that $\vec{R}_{CM}$ will lie on the axis joining their centers (let's call the line joining all three centers as $x$-axis and the line perpendicular to this line as $y$-axis). If we use the polar coordinate then we have
$$
R_{CM} = \frac{1}{M_{tot}} 2\int \int r \cos \theta \sigma   dA\\
\text{(I have written $2r\cos \theta$ because that's the thing we would get when we add any two}\\
 \text{vectors in that encircled region, $\sigma$ is the mass per unit area, and $dA$ is the area element)}\\
R_{CM} = \frac{1}{M_{tot}}2 \sigma \int_{r=0}^{R} \int_{\theta=0} r \cos \theta ~dA
$$
But the problem is that I don't know the upper limit of $\theta$, I worked hard but it seemed a little different in this case.
We can use our ordinary cartesian system,
$$
R_{CM} = \frac{1}{M_{tot}} 2\sigma \int \int x dx dy$$
the limit of $x$ will be (I think) $0$ to $R$ and we can get $y$ as
$$
(x-R/2)^2 + y^2 = R^2/4 \\
y= \sqrt{ x^2 - Rx}$$
So, we have
$$
R_{CM} = \frac{1}{M_{tot}} 2\sigma \int_{x=0}^{R} \int_{y=0}^{\sqrt{x^2-Rx}} x dy dx\\
R_{CM} = \frac{1}{M_{tot}} 2\sigma \int_{x=0}^{R} x\sqrt{x^2-Rx} ~dx
$$
I don't know how to carry out that last integral.
The answer to this question is "$R/6$ to the left of O" but where am I mistaking? Can someone help me out?
 A: I am assuming the mass per unit area is one for simplicity.
Using symmetry there is no need to actually compute an integral other than to determine masses.
In general, if you have two essentially disjoint sets $A,B$ then
$\int_{A \cup B} f = \int_A f + \int_B f$.
If we let $\bar{x}_A = {1 \over \int_A dm } \int_A x dm$, then
$\bar{x}_{A \cup B} = { \int_A dm \over \int_{A \cup B} dm } \bar{x}_A + { \int_B dm \over \int_{A \cup B} dm } \bar{x}_B$.
Furthermore, if we translate an object by $d$ we have
$\bar{x}_{A +\{d\}} = \bar{x}_A + d$.
Let $A$ be the big disc with the little disk cut out and let $B$ be the little disk.
If we take the centre of the big disk at the origin, we have
$\bar{x}_{A \cup B} = 0$, $\bar{x}_B = ({R \over 2},0)$,
$\int_{A \cup B} dm = \pi R^2$, $\int_{B} dm = {1 \over 4} \pi R^2$ (and $\int_{A} dm = {3 \over 4} \pi R^2$, of course).
Solving the above for $\bar{x}_A$ gives
$\bar{x}_A = -({R \over 6},0)$.
A: There is a simple solution, showing where the mass center of two bodies is located, knowing their position and their masses. (Well, on object has negative mass in our case.)
This answer addresses the question in the OP, it is closer to it (instead of the above physics approach) since it deals with the mentioned integral. We will "compute" it. (But not via polar transformation. But rather in a way fitting to the above intuition. However, it is a mathematical way to deal, since we use the linearity of the integral and proceed blindly.)
I will introduce some notations. Let $D$ be the (bigger) disc / the ball $B(0,R)$ centered in $0\in\Bbb C$ and with radius $R$, $D'$ be the (smaller) disk $B(R/2,R)$ centered in $R/2\in\Bbb C$ and with radius $R/2$, and $D''=D-D'$ the difference set.
(We have positioned the three centers in this way without loss of generality.)
We denote by $m,m',m''$ the total mass of the three sets. For symmetry reasons, the mass center of $D''$ is on the $Ox$ axis. Let $a=0$, $a'=R/2$, $a''=?$ be the three abscissa of the mass centers.
Then, in the spirit of the OP, where integrals are involved:
$$
\begin{aligned}
a'' 
&=
\frac 1{m''}\int_{D''}x\; dx\; dy\\
&=
\frac 1{m''}\int_{D}x\; dx\; dy
-
\frac 1{m''}\int_{D'}x\; dx\; dy\\
&=
\frac 1{m''}\cdot ma
-
\frac 1{m''}\cdot m'a'
\\
&=\frac m{m''}a-\frac {m'}{m''}a'
\\
&=\frac {1}{3/4}\cdot 0-\frac {1/4}{3/4}\cdot(R/2)
\\
&=\frac 16R\ .
\end{aligned}
$$
A: Here is another "physics oriented" way:
Because of symmetry the center of mass lies on the line connecting $O$ and $O'$. The mass at each point $-R \leq r \leq R$ on this line lies symmetric above and below which corresponds to a linear density
$$\rho(r) = 2\sqrt{R^2-r^2} \text{ for } -R\leq r\leq 0$$
and taking out the hole
$$\rho(r) = 2\left(\sqrt{R^2-r^2} - \sqrt{\left(\frac R2\right)^2-\left(r-\frac R2\right)^2}\right) \text{ for } 0\leq r\leq R$$
Now, noting that the mass of the disk with hole is $\frac 34 \pi R^2$ we get
$$r_M = \frac 4{3\pi R^2}\int_{-R}^Rr\rho(r) dr = - \frac 4{3\pi R^2}\int_0^Rr\sqrt{\left(\frac R2\right)^2-\left(r-\frac R2\right)^2}dr $$
$$= - \frac 4{3\pi R^2}\cdot \frac{\pi R^3}{8}=-\frac R6$$
A: The center of mass can probably be computed by using difficult integrals, but it can also be computed in a very simple way.
Let's call $\vec{OG}$ the center of mass, $m_{big}$ and $m_{small}$ the mass of the big and small disk respectively. By using the center of mass definition, we have,
$$ \vec{OG} = \frac{1}{m_{big} + m_{small}}(m_{big} \vec{OO} + m_{small} \vec{OO'}) $$
Since the small disk is cut out and not added we should not use a positive mass, but a negative mass for $m_{small}$, therefore, since the surface of the hole is 4 times smaller than the total surface,
\begin{align}
m_{small} &= -\frac{1}{4} m_{big}
\end{align}
By substituting this value into the first equation, we obtain,
$$\vec{OG} = - \frac{1}{3}\vec{OO'}$$
A: Place a Cartesian Coord. system, for example, at the center of the big disk. The mass of an homogeneous disk is $M=\rho \: A t$ where $\rho$ is the density, $A$ is the area of the circle and $t$ is the thickness.
You want $M\: z= \sum_i m_i z_i$ for $i=1,2$, where $m_1$ is the mass of the large disk and $m_2$ is the mass of the small disk, which is negative as you are removing it and $M=m_1-m_2$ is the total mass of the resulting system. The values of $z_i$ are the position in the $x_i$ and $y_i$ position of the small masses and $z$ is the position you are looking for.
Focus only in $x$ direction as $y$ is trivial. Compute the masses of the disks $m_1$ and $m_2$  (example: $m_i= \pi R*R t \: \rho$, you can use your integrals for this, but splitting the two regions makes it intuitive) and notice that the position of their centroids are $(x_1,y_1)=(0,0)$ and $(x_2,y_2)=(R/2,0)$. The answer is easy to find after algebraic operation.
A: Given the symmetry you noted, it seems not worth it to do the integral in polar coordinates when you know the center of mass will only have an $x$. Let's rotate your picture upside down. Notice that computing the integral will give us
$$\iint\limits_{\text{shaded region}}^{} x\:dA = \iint\limits_{\text{small circle on right side}}^{} x\:dA$$
by symmetry because $x$ is an odd function. Next, you want integrate w.r.t. $x$ first because the square roots will cancel
$$\bar{x} =\frac{4}{3\pi R^2}\int_{-\frac{R}{2}}^\frac{R}{2}  \int_{\frac{R}{2}-\sqrt{\frac{R^2}{4}-y^2}}^{\frac{R}{2}+\sqrt{\frac{R^2}{4}-y^2}} x\:dx \:dy = \frac{4}{3\pi R}\int_{-\frac{R}{2}}^\frac{R}{2} \sqrt{\frac{R^2}{4}-y^2}\:dy = \frac{R}{6}$$
