Why are $\left(\begin{smallmatrix}0&1\\0&0\\\end{smallmatrix}\right)$ and $\left(\begin{smallmatrix}0&0\\0&1\\\end{smallmatrix}\right)$ not similar? 
Why are the matrices $\begin{pmatrix}0 & 1 \\0 & 0 \\\end{pmatrix}$ and $\begin{pmatrix}0 & 0 \\0 & 1 \\\end{pmatrix}$ not similar? 

I can do a row operation. But on the other hand they don't have the same 
characteristic polynomial. How does it relates to: $A$ is similar to $B$ if and only if $A$ and $B$ have the same canonical form ?
 A: The first matrix, $A$, satisfies $A^{2}=0$.
For the second one, $B$, we see that its square does not equal $0$. This implies that we cannot have $A=S^{-1}BS$ for any $S$ because we would then have $A^{2}=S^{-1}B^{2}S$. 
A: The question seems to operate under the assumption that matrices which differ by elementary row operations should have the same characteristic polynomial.
This assumption is false (your matrices provide a counterexample). Elementary row operations preserve the row space and the kernel of a matrix, but not the eigenvectors.
A: What you're looking for is matrix similarity, not matrix equivalence.
Matrix equivalence merely says:
Are two matrices equivalent, you can transform them into each other, multiplying them with nothing but regular matrices. This is true exactly iff they have same rank.
A: Hint: Suppose they were equivalent then there had to exists a invertible matrix $S := \begin{pmatrix} a & b \\ c & d \end{pmatrix}$ such that
\begin{equation} \tag{1}
S^{-1} \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix} S = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}
\end{equation}
holds.
We have
$
S^{-1}
= \frac{1}{ad - bc} \begin{pmatrix} d & -b \\ -c & a \end{pmatrix}.
$
Therefore, $(1)$ becomes
\begin{align}
& \frac{1}{ad - bc} \begin{pmatrix} d & -b \\ -c & a \end{pmatrix}
\begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}
\begin{pmatrix} a & b \\ c & d \end{pmatrix}
= \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix} \\
\iff & \frac{1}{ad - bc} \begin{pmatrix} c d & d^2 \\ -c^2 & -c d \end{pmatrix}
= \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}
\end{align}
This yields $cd = 0$ and $-cd = 1$, which clearly is a contradiction.
