If both of $A,A^{-1}$ have entries from non negative integers then can we say $A$ is a permutation matrix? I've shown if both of $A,A^{-1}$ (assuming $A$ to be invertible) are $n\times n$ matrices with entries from natural numbers then both of them have to be permutation matrices. Now my question is if impose the condition that both of $A,A^{-1}$ have entries from non negative integers then can we say $A$ is a permutation matrix ?
 A: Let $u_1,\dots,u_n$ denote the rows of $A^{-1}$, and let $v_1,\dots,v_n$ denote the columns of $A$.  Note that $A^{-1}A = I$, so that $u_i \cdot v_i = 1$ for all $i$ and $u_i \cdot v_j = 0$ for all $i \neq j$.  Suppose that both have non-negative entries.
Claim: Each column of $A$ contains at most one non-zero entry.
Proof: Suppose that $v_j$ contains two non-zero entries $v_j(k_1) = A_{k_1,j}$ and $v_j(k_2) = A_{k_2,j}$.  Then $v_i \cdot u_j = 0$ for all $i \neq j$.  It follows that $u_i(k_1) = u_i(k_2) = 0$ for all $i \neq j$.  
However, this would imply that the vectors $\{u_i: i \neq j, \quad 1 \leq i \leq n\}$ fail to be linearly independent, which would imply that $A^{-1}$ fails to be invertible, which is a contradiction. $\square$
From there, note that $A$ not only has at most one non-zero entry per column, but that these entries must be equal to $1$.  Since $A$ is an invertible matrix in which every column contains exactly one non-zero entry equal to $1$, $A$ must be a permutation matrix.
It follows that $A^{-1}$ is a permutation matrix.
