A unit square has its corner chopped off to form a regular polygon with eight sides. What is the area of the polygon? 
A unit square has its corner chopped off to form a regular polygon with eight sides. What is the area of the polygon? Source: ISI BMATH UGA 2017 paper

A regular polygon with 8 sides can be divided into eight congruent triangles .I tried to find the area of a triangle in the following method. An angle of a triangle is 360/8=45..then I draw a perpendicular bisector of the angle which is height of the triangle  and I found it to be 1/2 as the square is of unit length .Now I got a right angled triangle from which I wanted to find the length of the base of the triangle but I couldn't do so.
 A: If the side of  the cut triangle is $x$, then in order to have a regular polygon, we need to have $$1-2x = \sqrt 2 x $$
solving for $x$ we get $$ x= 1- \frac {\sqrt 2 }{2}$$
The  total area of the cut is $2x^2$ and the area of polygon is  $$A=1-2x^2 =  2 (\sqrt 2 -1)  \approx 0.8284$$    
A: An image makes everything simpler to understand:

Because the square side length is 1,
$$b + a + b = 1$$
Solving for $b$ we get
$$b = \frac{1 - a}{2} \label{NA1}\tag{1}$$
Squaring this (noting that we are limited to positive $a$ and $b$, i.e. that $a \gt 0$ and $b \gt 0$) we get
$$b^2 = \frac{1 - 2 a + a^2}{4} \label{NA2}\tag{2}$$
In the corners, we need $b$ such that the hypotenuse is $a$. Pythagorean theorem says
$$b^2 + b^2 = a^2$$
solving for $b$ we get
$$b^2 = \frac{a^2}{2} \label{NA3}\tag{3}$$
Combining $\eqref{NA2}$ and $\eqref{NA3}$ we get
$$b^2 = \frac{a^2 - 2 a + 1}{4} = \frac{a^2}{2}$$
i.e.
$$2 a^2 = a^2 - 2 a + 1$$
which simplifies to
$$a^2 + 2 a - 1 = 0$$
and noting that the first two terms are from squaring $a$, to
$$a^2 + 2 a + 1 - 2 = (a + 1)^2 - 2 = 0$$
Moving the last term ($-2$) to the other side, we get
$$(a + 1)^2 = 2$$
so taking a square root on both sides, and remembering that we need $a \gt 0$, we get
$$a = \sqrt{2} - 1$$
Substituting this to $\eqref{NA1}$ we can solve $b$,
$$b = \frac{1 - a}{2} = \frac{1 - \sqrt{2} + 1}{2} = \frac{2 - \sqrt{2}}{2} = 1 - \sqrt{\frac{1}{2}}$$
Note also that
$$b^2 = \left(1 - \sqrt{\frac{1}{2}}\right)^2 = 1 - 2 \sqrt{\frac{1}{2}} + \frac{1}{2} = \frac{3}{2} - \sqrt{2}$$
The corner triangles total area is the same as the area of two $b$-sided squares. So, the area of the octagon is the area of the unit square (1) minus the area of two $b$-sided squares:
$$A_{octagon} = 1 - 2 b^2 = 1 - 2 (\frac{3}{2} - \sqrt{2}) = 1 - 3 + 2 \sqrt{2} = 2 \sqrt{2} - 2 \approx 0.828$$
A: Let's have a look at the corners we chopped off. It is a right triangle, because it was the corner of a square. It is an equilateral triangle because of symmetry of the resulting polygon. Its hypotenuse is the base of the triangle you are looking for.
Lets call the length of the hypothenuse $b$ and the length of each of the other two sides $a$, then by Pythagoras
$$b^2 = 2a^2$$
On the other hand, the side of your square is such a base of a triangle into which you divided the polygon + 2 times the length of the shorter side of the triangle you chopped of (that is, $a$), so if $c$ would be the length of each site of the original square, we get
$$c=b+2a$$
As I understand you, $c$ is given (I think you mentioned $c=1$ in your case) so you can effectively solve for $b$. (note that $2a^2 = 2(a^2) \neq (2a)^2$ in general)

Answering your question from your text aside, the figure you are trying to find the area for is an octagon, so it's area would be $2(1+\sqrt{2})b^2$ or, equally, be $2(\sqrt{2}-1)c^2$ (since $c$ is double the apothem, see the wikipedia link).
