Finding Area of the red square Calculate the area of the red square.
I m having some trouble doing it so I need some help.

 A: (with slightly different point labels)

Introduce $a$ as the radius of the big circle and $d$ as the side of the square. Take $O$ as the origin of the coordinate system with $x$ axis going to the right and $y$ axis going upwards. We have the foolowing point coordinates: 
$$H(x_H, y_H)$$
Note that $x_H>y_H$. Because of symmetry:
$$G(x_G,y_G)\equiv G(y_H,x_H)$$
So the side of the square is:
$$d=\sqrt{(x_G-x_H)^2+(y_G-y_H)^2}=\sqrt{(y_H-x_H)^2+(x_H-y_H)^2}=\sqrt{2}(x_H-y_H)\tag{1}$$
Coordinates of point $I$ are:
$$I(x_I,y_I)\equiv I(x_H+d\frac{\sqrt 2}{2},y_G)\equiv I(x_H+(x_H-y_H), x_H)\equiv I(2x_H-y_H, x_H)$$
Point $H$ is on the small circle so:
$$(x_H-\frac a2)^2+y_H^2=(\frac a2)^2\tag{2}$$
Point $I$ is on the big circle so:
$$(2x_H-y_H)^2+x_H^2=a^2\tag{3}$$
So you have a system with of two equations, (2) and (3), with two unknowns $(x_H,y_H)$. The system leads to the following quartic equation for $a=7$:
$$32x_H^4-56x_H^3-343x_H^2-686x_H+2401=0$$
In theory, you can solve it by hand. I did it with Mathematica:
$$x_H=4.40864,\ \ y_H=3.38$$
Other solutions had to be discarded (two of them are complex, the third one leads to negative value of $y_H$). From (1):
$$d=1.45471$$
...which is pretty much in sync with the value obtained by measuring the length of the square in Geogebra.
