First, some notation. For $a,b \in F^\times$, denote by $\left(\frac{a,b}{F}\right)$ the quaternion algebra over the field $F$ defined by
$$
i^2 = a, \qquad j^2 = b, \qquad ji = -ij \, .
$$
Given a quaternion algebra as above, we can form the "field extension" $K = F[i] = \frac{F[x]}{(x^2 - a)}$. I write "field extension" in quotes because if $a$ is a square in $F$, then $K$ is not a field: if $a = c^2$, then
$$
K = \frac{F[x]}{(x^2 - a)} = \frac{F[x]}{(x^2 - c^2)} \cong \frac{F[x]}{(x - c)} \times \frac{F[x]}{(x + c)} \cong F \times F.
$$
If $K$ is a field, we have the field norm
\begin{align*}
N_{K/F}: K &\to F\\
c_1 + c_2 i &\mapsto (c_1 + c_2 i)(c_1 - c_2 i) = {c_1}^2 - {c_2}^2 i^2 = {c_1}^2 - a{c_2}^2 \, .
\end{align*}
Theorem 5.4.4 (p. 63) of this forthcoming book by John Voight states multiple criteria for a quaternion algebra to be split. The one I will use is the following:
Proposition: Let $B=\left(\frac{a,b}{F}\right)$ be a quaternion algebra over a field $F$ with $\operatorname{char}(F) \neq 2$. Then
$$
B \cong M_2(F) \iff b \in N_{K/F}(K^\times)
$$
where $K = F[i]$. (If $K$ is not a field, we take $N_{K/F}(K^\times) = F^\times$.)
For extensions of finite fields the norm map is surjective (see here or here, for example), which implies, together with the above theorem, that every quaternion algebra $B$ over a finite field is split.
We can find this isomorphism to $M_2(\mathbb{F}_p)$ more explicitly as follows. For any $a \in \mathbb{F}_p^{\times}$, the quaternion algebra $B_0 = \left(\frac{a,1}{\mathbb{F}_p}\right)$ is split via the isomorphism
\begin{align*}
i &\mapsto
\begin{pmatrix}
0 & a\\
1 & 0
\end{pmatrix}\\
j &\mapsto
\begin{pmatrix}
1 & 0\\
0 & -1
\end{pmatrix} \, .
\end{align*}
Given $a,b \in \mathbb{F}_p^{\times}$, we will show that $B = \left(\frac{a,b}{\mathbb{F}_p}\right)$ is isomorphic to $B_0$.
Since the norm map $N = N_{K/\mathbb{F}_p}$ is surjective, then there exists $\alpha = c_1 + c_2 i \in K$ such that $N(\alpha) = \frac{1}{b}$. Observe that
$$
j \alpha = j (c_1 + c_2 i) = c_1 j + c_2 ji = c_1j - c_2 ij = (c_1 - c_2 i)j = \overline{\alpha} j \, .
$$
Letting $J = \alpha j$, then $1, i, J, i J$ is a basis for $B$, and
\begin{align*}
J^2 = (\alpha j)^2 = \alpha j \alpha j = \alpha \overline{\alpha} j^2 = N(\alpha) b = \frac{1}{b} b = 1
\end{align*}
Thus $B \cong \left(\frac{a,1}{\mathbb{F}_p}\right)$.
See also Exercise 3.13 on p. 39, which outlines a different proof of the result using group theoretic techniques.
