Recall that a system can have either $0$, $1$, or infinitely many solutions. Thus, the fact that there is at least one nontrivial solution (other than the trivial solution consisting of the zero vector) implies that there are infinitely many solutions. Thus, your statement is false; as a counterexample, consider the folloring homogeneous augmented matrix (conveniently in reduced row echelon form):
1 & 0 & 2 & 0 \\
0 & 1 & 3 & 0
Notice that $A$ has infinitely many solutions (the third column has no pivot, so the system has one free variable), yet there is no row of zeroes.
Note: The converse is not necessarily true either. That is, it is NOT the case that: if the row echelon matrix of a homogenous augmented matrix A has a row of zeroes, then there exists a nontrivial solution. As a counterexample, consider:
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 0
Notice that $A$ has only the trivial solution (every column has a pivot, so the system has no free variables), yet $A$ has a row of zeroes.