Is the following generalization of Strong Law of Large Numbers valid? According to SLLN, if $X_1, X_2, \ldots$ is an infinite sequence of i.i.d. random variables with expected value $\mu$ and $S_n := \sum_{i=1}^n X_i/n$ then
$S_n \to \mu$ almost surely.
If the sequence is instead $X_{1,1}, X_{2,1}, X_{2,2}, \ldots, X_{n,1}, \ldots,X_{n,n}, X_{n+1,1}, \ldots$ whose terms are i.i.d. random variables with expected value $\mu$ and $S_n := \sum_{i=1}^n X_{n,i}/n$, can we still say $S_n \to \mu$ almost surely? If not are there any extra conditions that make this true?
I will give an example for why this is not trivial. Consider distribution of variables is Bernoulli with $p=1/2$ and the sample 
$0, 1, 1, 0, 0, 0, 1, \ldots$. That is $(2n-1) \times 0$ are followed by 
$2n \times 1$ and that is repeated for every $n > 0$. For this sequence $S_n$ converges to $1/2$ in the first case and the limit does not exist in the second.
If the probability of all such examples is $0$ then convergence will be almost sure, but this is something that can probably be proven on a case by case with union bound inequality. I just wonder whether there is some general result that makes such analysis easier.
 A: First observe that in this setting, the sequence $\left(S_n\right)_{n\geqslant 1}$ is independent. For an independent sequence, in view of the Borel-Cantelli lemma, almost sure convergence and complete convergence are equivalent. Hence $S_n\to \mu$ almost surely if and only if for all positive $\varepsilon$, 
$$\tag{*}
\sum_{n\geqslant 1}\mathbb P\left(\lvert S_n-\mu\rvert \gt\varepsilon\right)<+\infty.
$$
Let $(Y_i)_{i\geqslant 1}$ be an i.i.d. sequence such that $Y_1$ has the same law as $X_{1,1}$. Then $(*)$ is equivalent to 
$$
\forall \varepsilon>0, \sum_{n\geqslant 1}\mathbb P\left(\left\lvert \sum_{j=1}^n(Y_j-\mu)\right\rvert \gt n\varepsilon\right)<+\infty.
$$
By Theorem 3 in this paper by Baum and Katz, this is equivalent to $\mathbb E[Y_1^2]<\infty$, hence we do need extra conditions.
A: Yes, the result holds equally well in both cases. More generally, Let $(X_i:i=1,2,...)$ be any infinite sequence of i.i.d. random variables with expected value $μ$, and let $F: \Bbb Z_+\to\mathrm P(\Bbb Z_+)$ such that $|F(k)|\in\Bbb Z_+$ with $|F(k)|\to\infty$ as $k\to\infty$. Then$$\frac1{|F(n)|}\sum_{i\in F(n)}X_i\;\to_{\text{a.s.}}\;\mu$$as $n\to\infty$. The proof is essentially the same as that for the simple average of an initial segment of the original sequence, because the i.i.d. property of any (finite or infinite) subsequence of the original sequence $(X_i:i=1,2,...)$ follows from that of the original sequence.
