A complex matrix commuting with all diagonalizable matrices is a scalar matrix This question is from an old comprehensive exam

Let $A$ be an $n\times n$ complex matrix, which commutes with all diagonalizable $n\times n$ complex matrices. Prove that $A$ is a constant times the identity matrix. Hint: What matrices commute with all diagonal matrices?

The hint says to find the matrices that commute with all diagonal matrices, which should themselves be diagonal.
What I've tried: Let $A \in M_n(\mathbb{C})$, $B$ be a diagonalizable $n\times n$ complex matrix  such that $P^{-1}BP = D$, a diagonal matrix.
As $AB =BA$ then $P^{-1}ABP =P^{-1}BAP$, so
$$P^{-1}APP^{-1}BP =P^{-1}BPP^{-1}AP$$
Which means that
$$(P^{-1}AP)D = D(P^{-1}AP)$$
And so $P^{-1}AP$ is diagonal, so $A$ is diagonalizable, and moreover is simultaneously diagonalizable.
But from here I'm stuck. I know that $A$ commutes with all diagonalizable matrices, so this should always work, but I don't see how to get that $A$ is a scalar matrix.
Am I missing something obvious? Thank you in advance for your help.
 A: This is just a matter of grinding through:
let $\Lambda = \operatorname{diag} \{1,...,n \}$ and compute
$[\Lambda A - A \Lambda]_{ij}$ to show that $A$ is diagonal.
Pick $i \neq j$ and let $V=e_i e_j^T + e_j e_i^T$. Note that $V$ is symmetric hence diagonalisable and compute
$VA-AV$ to show that $\lambda_i=\lambda_j$.
A: Let's assume that you can show that only a diagonal matrix commutes with all diagonal matrices (this is a triviality, by comparing matrix coefficients). Now notice that matrix that commutes with all matrices must be diagonal with respect to all bases (conjugate and repeat the previous argument). This means that every vector is an eigenvector, which, in turn, obviously means that the matrix is a multiple of the identity.
A: So, the hint points out that if $A $ commutes with every diagonalizable matrix,  then it commutes with every diagonal matrix.
Look at the $2×2$ case.  We get $A $ commuting with $\begin{pmatrix}1&0\\0&0\end{pmatrix} $.  But that means if $A=\begin  {pmatrix}a&b\\c&d\end {pmatrix} $, we get $\begin{pmatrix}a&0\\c&0\end {pmatrix}=\begin{pmatrix}a&b\\0&0\end{pmatrix} $.  So we know $A $ is diagonal. (In the general case, use the matrix with a  $1$ on the $jj $-th entry and $0$'s everywhere else to show that the  $j $-th row and $j$- th column are zero off the diagonal.)
Next, I claim $a=d $.  Because we know $\begin  {pmatrix}0&a\\d&0\end {pmatrix}=A\begin {pmatrix}0&1\\1&0\end {pmatrix}=\begin  {pmatrix}0&1\\1&0\end {pmatrix}A=\begin  {pmatrix}0&d\\a&0\end {pmatrix}$.
Now you just need to generalize to the  $n×n $ case.
