Constructing the $Z_2 \times Z_2$ group table In A. Zee's group theory book p. 47-49, he constructs the group table with four elements $\{I,A,B,C\}$
$\begin{array}{c|cccc}
& I & A & B & C \\ 
\hline
I & I & A & B & C \\ 
A & A \\ 
B & B \\ 
C & C
\end{array}$
There are two possibilities for the second row second column, $A^2 = I$ or $A^2 = B$ (it could also be $~A^2 = C~$ but your $~C~$ is my $B~$).
For $A^2 = B$, it corresponds to the $Z_4$ group, and the construction of the group table is clear,
$\begin{array}{c|cccc}
& I & A & B & C \\ 
\hline
I & I & A & B & C \\ 
A & A & B & C & I \\ 
B & B & C & I & A \\ 
C & C & I & A & B
\end{array}$
For $A^2 = I$, the second row and second column is a consequence of the "once and only once rule" of the group table,
$\begin{array}{c|cccc}
& I & A & B & C \\ 
\hline
I & I & A & B & C \\ 
A & A & I & C & B \\ 
B & B & C \\ 
C & C & B 
\end{array}$
Now, the lower right part of the table has two possibilities
$\begin{array}{c|c}
I & A \\ 
\hline
A & I
\end{array}$
$\begin{array}{c|c}
A & I \\ 
\hline
I & A
\end{array}$
Zee's says that these two amount to the same thing, you just rename $B$ and $C$, which when you check the group composition, either one is consistent so that we have,
$\begin{array}{c|cccc}
& I & A & B & C \\ 
\hline
I & I & A & B & C \\ 
A & A & I & C & B \\ 
B & B & C & I & A \\ 
C & C & B & A & I
\end{array}$
but what does the statement "these two amount to the same thing, you just rename $B$ and $C$" really mean? I need more exposition to understand more what is the argument (BTW, this is the $Z_2 \times Z_2$ group ).

There is another question, we can construct the $Z_2 \times Z_2$ group table through the cyclic subgroups,
If $~A^4 = I~$ then the group is $Z_4$. By Lagrange's theorem, $~A^3 = I~$ is not possible since a group with order four cannot have a subgroup of order three. If $A^2 = I$ then we multiply $B$ by itself, either $B^2 = I$ or $B^4 = I$. The latter is ruled out so we have $B^2 = I$ and $AB = BA = C$.
I need some clarifications on why,

*

*We multiply $B$ by itself. Is it because $A$, $A^2$ is already cyclic so that we test if it is consistent if $B$ is also a cyclic subgroup? Is it really necessary that all subgroups must be cyclic if one of it is cyclic?

*$B^4 = I$ is ruled out. I cannot find the argument on why this must be the case. I know I need to do some group element multiplication and find a contradiction but so far I cannot find any.

 A: The two choices for the lower right quadrant do not amount to the same thing.  The first choice results in the non-cyclic group which you call $Z_2 \otimes Z_2$ (I would call it $C_2 \times C_2$).  The second one results in a cyclic group ($Z_4$, or $C_4$).  You can match it with your $Z_4$ table by relabeling $A$ and $B$.  (As Arthur pointed out in the comments, you did a similar relabeling earlier when you said "your $C$ is my $B$".)  You can tell that these groups are really different because the first has the property that every element squares to $I$, and the second doesn't.
As for the second half of your question, let me first point out that it's not actually true that if $A^4 = I$, then the group is cyclic.  In fact, even in the non-cyclic group, you also have $A^4 = I$; it's just that $A^2 = I$ too.  What you mean to say is that if $A$ has order 4, then the group is cyclic. (In case you haven't seen this term yet, the order of $A$ is defined as the smallest $n > 0$ with the property that $A^n = I$.)
Now let me try to clarify how to do the classification in terms of cyclic subgroups.  As you said, Lagrange's theorem implies that every element has order 1, 2, or 4.  (And of course only the identity, $I$, has order 1.)  If there is any element with order 4, then that makes the group cyclic.  (It could be $A$, $B$, or $C$--and in fact it will be two out of those three--but as before you can rename elements so that $A$ is one of them.)  If there is no element with order 4, then everything but $I$ must have order 2.  This means that you have all $I$'s along the diagonal of your multiplication table.  Given this, you can fill in the rest of the table using the "once and only once rule" that you mentioned.  This proves that every group of order 4 is either $C_4$ or $C_2 \times C_2$.
