If you really want to see a "pattern" then you need to write the multiplicative group down in "cyclic order" as powers of two: $2^0=1$, $2^1=2$, $2^2=4$, $2^3=8=3$,
back to $2^4=1$. Then the inverses go in the opposite order $2^{-0}=1$, $2^{-1}=2^3$, $2^{-2}=2^2$, $2^{-3}=2^1$.
So in table format
$$
\begin{array}{|c|c|c|c|}
\hline
\text{of}& 0 & 1 & 2 & 4 & 3 \\ \hline
\text{is}& \not\exists& 1& 3 & 4 & 2\\ \hline
\end{array}
$$
or
$$
\begin{array}{|c|c|c|c|}
\hline
\text{of}& 0 & 1 & 2 & 4 & 3 \\
\text{power form}&\not\exists&2^0&2^1&2^2&2^3\\ \hline
\text{is}& \not\exists& 1& 3 & 4 & 2\\
\text{power form}&\not\exists&2^4&2^3&2^2&2^1\\ \hline
\end{array}
$$
Every finite field has so called primitive elements (at least one) $g$ such that all the non-zero elements of the field are of the form $g^i$ with $g=0,1,2,\ldots,q-2$ where $q$ is the number of elements of the fields. Here $g^0=g^{q-1}=1$. The above table was based on the choice $g=2$, but we could have used $g=3$ equally well. For the field $\Bbb{Z}_7$ we can use $g=3$ or $g=5$, for the field $\Bbb{Z}_{11}$ we can use $g=2$. Finding a primitive element is not always trivial, and there are difficult open questions. For example, it is not known whether $g=2$ is a primitive element of the field $\Bbb{Z}_p$ for infinitely many primes $p$. See Artin's conjecture.
To answer the question in the title. The extended Euclidean algorithm is the simplest to understand, fast method for finding inverses in fields of the form $\Bbb{Z}_p$. Basically it is a quick method for finding the integers $a,b$ that appear in Elvorfirilmathredia's answer. For heaven's sake don't try to find a primitive element and build tables like those above when $p$ is, say, a hundred digit prime :-)
There is no way to even "approximate" the difference
between $1/x$ and a "linear pattern" like $ax+b$ ("approximate" in a sense that I would need to spend a few hours talking about). Basically, in average the "error" is jumping all over the place. The way I would quantify this claim would be based on Kloosterman sums.