If $G$ is NOT full rank, does $A = G'G$ always singular? I know that for matrix with full rank, $G$. Matrix $A = G'G$ is non-singular. My question is, does the converse also hold?   
 A: The rank of $G'G$ is equal to the rank of $G$. (One way to see this is to show that the two matrices have the same nullspace: $Gx=0 \iff G'Gx=0$.)
A: Yes.  If $G$ is a square matrix, $G$ not of full rank implies $\det(G) = 0$, whence $\det(A) = \det(G') \det (G) = 0$, so $A$ must be singular.
A more general argument works even when $G$ is not square, and it goes like this:  writing $G$ in terms of its columns, like this
$G = [g_1 \; g_2 \; . . . \; g_n], \tag{1}$
so that $g_1$ is the first column of $G$, $g_2$ the second column, and so forth, it is easy to see from the ordinary rules of matrix multiplication that, for any matrix $F$ such that $FG$ is meaningful (that is, primarily, the column size of $F$ is equal to the row size of $G$), that with $G$ as in (1), 
$FG = [Fg_1  \; Fg_2 \; . . . \; Fg_n];  \tag{2}$
now if $G$ is of less than full rank, there is a linear dependence amongst its columns, so there are scalars $\alpha_j$, not all zero, with
$\sum \alpha_j g_j = 0; \tag{3}$
applying $F$ to (3) yields
$\sum \alpha_j Fg_j = 0, \tag{4}$
exhibiting a non-trivial linear dependency 'twixt the columns of $FG$ as well; hence, $FG$ is of less than full rank.  Taking $F = G'$ polishes off the result for the present specific case.
Hope this helps.  Cheerio,
and as always,
Fiat Lux!!!
