This statement holds true when the Jordan blocks of A associated with eigenvalue 0 are all 1 by 1; Otherwise, there would be extra rank entailed by such Jordan blocks. For example, $rank(J_2(0)) = 1$.
Equivalently you can say the geometric and algebraic multiplicity of eigenvalue 0 agrees with each other.
Or, the minimal polynomial of A is $q_A(t)=t*\Pi_{i=1}^{d-1}(t-\lambda_i)^{r_i}$, where we assumed there are d distinct eigenvalues of A and the maxiaml sizes of their corresponding Jordan blocks are $r_i$.
Note that I took nonzero eigenvalue of algebraic mulitplicty k>1 as k nonzereo eigenvalues instead of 1; Otherwise we would need the equivalence between algebraic and geometric multiplicity of not only zero eigenvalues but also that of the nonzero eigenvalues of A. And therefore the diagonalizability would be necessary.
I could be wrong, feel free to correct me.