Suppose that $A=I+3B$ is torsion. Then, note that it's eigenvalues must all be of unit length. But, the eigenvalues of $I+3B$ are precisely those of the form $1+3\lambda$ for $\lambda$ an eigenvalue of $B$. So, in particular, we see that for any eigenvalue $\lambda$ of $B$ that
$$|3\lambda|=|1+3\lambda-1|\leqslant |1+3\lambda|+1=2$$
So, $|\lambda|\leqslant \frac{2}{3}$. But, note that since all the conjugates of $\lambda$ must also be roots of the characteristic polynomial of $B$ (since the characteristic polynomial is in $\mathbb{Z}[x]$), the same argument shows that they must also have norm less than or equal to $\frac{2}{3}$. So, if $m(x)$ denotes the minimal polynomial for $\lambda$ over $\mathbb{Q}$, then it's constant term is an integer (since $\lambda$ is an algebraic integer), but certainly less than $\frac{2}{3}$ in modulus, being (up to sign) the product of the conjugates of $\lambda$, and so must be $0$. But, since $m(x)$ is irreducible, and monic, this implies that $m(x)=x$. So $\lambda=0$. So, all the eigenvalues of $B$ are $0$.
But, then this implies that all the eigenvalues of $A$ are $1$, but since $A$ is diagonalizable, since it's finite order, $A=I$.