Quadratic reciprocity in the case $a=-1$

I am reading the proof the for odd prime $p$, $$\left ( \frac{-1}{p} \right)_2 = (-1)^{\frac{p-1}{2}} = \begin{cases} 1 \hspace{2mm} \text{for} \hspace{2mm} p \equiv 1 \operatorname{mod} 4 \\ -1 \hspace{2mm} \text{for} \hspace{2mm} p \equiv 3 \operatorname{mod} 4 \end{cases}$$

The proof states that if $-1$ is a square mod $p$, then a square root of it has order $4$ in $(\mathbb{Z}/p)^{\times}$. That is, if $-1 = b^2 \operatorname{mod} p$, then one of the possible square roots of $b$ generates a subgroup of order $4$ in $(\mathbb{Z}/p)^{\times}$.

Why is this true? (Kinda makes sense if we square both sides of the equation but I am not sure this is allowed)

No, if $-1\equiv b^2\bmod p$, then $b$ itself generates a subgroup of order $4$ in $(\mathbb{Z}/p\mathbb{Z})^\times$, namely $$\langle b\rangle\;=\;\{b^0=1,\;\;b^1=b,\;\;b^2=-1,\;\;b^3=-b\}$$ After that, we get $b^4=(-b)\cdot b=(-1)\cdot (-1)=1$. Therefore $\langle b\rangle$ has $4$ elements.
If $b$ has a square root (which need not be the case) then it would generate a subgroup of order $8$.