Permutations avoiding repeated elements in a row Assume a working set of: {A, B, C}
I'm creating a new set of 10 elements using the working set. The only restriction is that no single element can repeat more than 4 in a row.
For example:
Valid: AAABBCABAA
Invalid: AAAAABCABC
Valid: AAAABCABCA
Invalid: BABAACCCCC
How many permutations are possible with new set size of 10 using the working set without violating this rule?
How many permutations exist for variable ranges of set sizes using the working set? For example, how many combinations exist that do no have any element repeat from the working set for sizes 8 through 10?
 A: Let $D(n)$ be the number of strings of length $n$ that have a single item at the end, $E(n)$ the number of strings of length $n$ that have two of the same at the end, and $F(n)$ the number of strings of length $n$ that have three of the same at the end.  We have $D(1)=3, E(1)=F(1)=0$ and the recurrence $D(n)=2(D(n-1)+E(n-1)+F(n-1)), E(n)=D(n-1), F(n)=E(n-1)$ because you can put two letters on any string to make a D and one letter to extend the last run to make an E or F.  You could solve this analytically but for small lengths it is easier in Excel.  I find $5676$ strings of length $8$, $48384$ of length $10$
A: The following answer is long but introduces what I feel is a wonderfully elegant method which itself introduces the use of regular expressions to find the generating function that enumerates valid strings. 

Let's imagine listing all the different valid strings and let's separate the strings with "+"s. We will call this list a regular expression $R$. The list will begin
$$\begin{align}
R=\epsilon &+ A + B + C &\\
&+ AA + AB + AC + BA + BB + BC + CA + CB + CC &\\
&+ AAA + AAB + AAC + ABA + ABB + ABC + ACA + ACB + ACC &\\
&+ BAA + BAB + BAC + BBA + BBB + BBC + BCA + BCB + BCC &\\
&+ CAA + CAB + CAC + CBA + CBB + CBC + CCA + CCB + CCC &\\
&+\cdots +AAAAB + AAAAC + \cdots &
\end{align}$$
Where $\epsilon$ is the empty string.
Now lets have some rules:


*

*We will allow ourselves to factorise regular expressions (whilst maintaining order) then, for example, we may write $AC+BC$ as $(A+B)C$ or we may write $CA+CB$ as $C(A+B)$ but $C(A+B)\ne (A+B)C$, also we cannot factor $AC+CA$. 

*It doesn't matter in which order we write the list so for example $A+B=B+A$. 

*We can define the removal of strings from the list with subtraction, so for example $A+B+AB +AA-AB =A+B+AA$.

*We will take powers of letters to mean multiple consecutive occurrences E.g. $A^2B^3C^2=AABBBCC$.
With these rules we can manipulate our regular expressions. 
First, split up our regular expression $R$ into regular expressions $R_A$, $R_B$ and $R_C$ consisting of all of those valid strings ending with $A$, $B$ and $C$ respectively hence
$$R=\epsilon + R_A + R_B + R_C$$
Then any string in our regular expression $R_A$ can only be expressed as a valid string that ends with something other than $A$ followed by either $1$,$2$, $3$ or $4$ consecutive $A$s we may write this
$$R_A=(R-R_A)A+ (R-R_A)AA+(R-R_A)AAA+(R-R_A)AAAA=(R-R_A)(A+A^2+A^3+A^4)$$
similarly
$$R_B=(R-R_B)(B+B^2+B^3+B^4)$$
$$R_C=(R-R_C)(C+C^2+C^3+C^4)$$
As it stands this is the most that can be achieved with regular expressions but we must recognise what it is that we want to achieve: we only want to count the valid strings of length $k$. 
In other words we don't care about the order of the letters or about distinguishing them, we may as well label them all $x$. 
Now we know that order within a string doesn't matter we can treat our regular expressions exactly like equations in our variable $x$ so that
$$R=\epsilon + R_A + R_B + R_C \quad \text{becomes}\quad r(x)=1+r_a(x)+r_b(x)+r_c(x)\tag{1}\label{1}$$
and
$$R_A=(R-R_A)(A+A^2+A^3+A^4)\quad \text{becomes}\quad r_a(x)=(r(x)-r_a(x))(x+x^2+x^3+x^4)$$
$$R_B=(R-R_B)(B+B^2+B^3+B^4)\quad \text{becomes}\quad r_b(x)=(r(x)-r_b(x))(x+x^2+x^3+x^4)$$
$$R_C=(R-R_C)(C+C^2+C^3+C^4)\quad \text{becomes}\quad r_c(x)=(r(x)-r_c(x))(x+x^2+x^3+x^4)$$
these may be rearranged to give
$$r_a(x)=r_b(x)=r_c(x)=\frac{r(x)(x+x^2+x^3+x^4)}{1+x+x^2+x^3+x^4}$$
which can be substituted into $\eqref{1}$ to give
$$r(x)=1+3\frac{r(x)(x+x^2+x^3+x^4)}{1+x+x^2+x^3}$$
$$\implies r(x)=\frac{1+x+x^2+x^3+x^4}{1-2(x+x^2+x^3+x^4)}\tag{2}\label{2}$$
this is our generating function for valid strings and has the expanded form 
$$r(x)=\sum_{k=0}^{\infty}r_kx^k$$
As you can see the exponent on the $x$ enumerates the valid string lengths and $r_k$ is the number of valid strings length $k$.
You have asked for the coefficients $r_8$, $r_9$ and $r_{10}$. 
We can either use a computer algebra system such as sage to expand the generating function for us or we can manipulate it into a form that will give us a recurrence for $r_k$.
Using sage we simply input
r(x)=(1+x+x^2+x^3+x^4)/(1-2*(x+x^2+x^3+x^4)) 
show(taylor(r(x),(x,0),10))

which returns the Taylor expansion about $x=0$ of $r(x)$ up to the $x^{10}$ term:
$$55896 \, x^{10} + 18792 \, x^{9} + 6318 \, x^{8} + 2124 \, x^{7} + 714 \, x^{6} + 240 \, x^{5} + 81 \, x^{4} + 27 \, x^{3} + 9 \, x^{2} + 3 \, x + 1$$
As you can see
$$r_8=6318$$
$$r_9=18\,792$$
$$r_{10}=55\,896$$
Or, the recurrence which I mentioned above can be found by rearranging $\eqref{2}$
$$r(x)= 1+x+x^2+x^3 +x^4 + 2(x+x^2+x^3+x^4)r(x)$$
$$\implies\sum_{k=0}^{\infty}r_kx^k=1+x+x^2+x^3+x^4+\sum_{k=0}^{\infty}2(r_{k-1}+r_{k-2}+r_{k-3}+r_{k-4})x^k$$
$$\implies r_k=2(r_{k-1}+r_{k-2}+r_{k-3}+r_{k-4})+\sum_{j=0}^{4}\delta_{j,k}$$
where
$$\delta_{j,k}=\begin{cases}1 & j=k\\0 & j\ne k\end{cases}$$
A lovely tool for listing terms using a recurrence is Microsoft Excel because we can easily use formulae to relate cells in a column to previous cells in said column.
A third route is also open to us: we may factorise the denominator of $r(x)$ then express as partial fractions
$$\begin{align}r(x)&=\frac{1+x+x^2+x^3+x^4}{(1-\phi_1x)(1-\phi_2x)(1-\phi_3x)(1-\phi_4x)}\\&=(1+x+x^2+x^3+x^4)\left(\frac{A_1}{1-\phi_1x}+\frac{A_2}{1-\phi_2x}+\frac{A_3}{1-\phi_3x}+\frac{A_4}{1-\phi_4x}\right)\end{align}$$
which will eventually yield a closed for for $r_k$, however this requires us to solve a quartic which is, in general, a messy business so I will leave that to the reader.
