$p$-Sylow subgroups of $SL_4(\mathbb{F}_p)$

Let $p$ be a prime. Then $$|SL_4(\mathbb{F}_p)| = p^6(p^2+1)(p^3-1)(p^2-1)(p-1)$$ Also, let $P$ be the collection of upper triangular matrices with $1$'s on the diagonal. Then $P$ is a group, and $|P| = p^6$, and therefore $P$ is a $p$-Sylow subgroup of $SL_4$.

Now I know that the normalizer of $P$ is the upper triangular matrices with a unity determinant. This means that $|N_G(P)| = (p-1)^3p^6$. I need to find the number of $p$-Sylow groups of $SL_4$. Since $p$-Sylow groups are conjugate, we can write $$n_p = \left|\{APA^{-1}\ |\ A\in SL_4\}\right|$$ We can also write $$\{APA^{-1}\ |\ A\in SL_4\} = \{APA^{-1}\ |\ A\in N_G(P)\}\cup\{APA^{-1}\ |\ A\in SL_4\setminus N_G(P)\}$$ $$= \{P\}\cup\{APA^{-1}\ |\ A\in SL_4\setminus N_G(P)\}$$ The problem is that I have failed to count how many elements in $\{APA^{-1}\ |\ A\in SL_4\setminus N_G(P)\}$ satisfy $APA^{-1} = BPB^{-1}$. This happens if and only if $B^{-1}A\in N_G(P)$. How can I deduce how much such matrices exist?

• Am I missing something? The number of Sylow $p$-subgroups equals $|SL_4(\mathbb{F}_p ): N_G(P)|=(p^2+1)(p^2+p+1)(p+1)$. – Nicky Hekster Dec 18 '17 at 8:42