True Randomness and repetition I do not have a degree in any field of mathematics; however I would like to get an input perhaps from those who do.
I argued a point with one of my children the other day that if all of the arguments against a system that could generate true random numbers were moot (algorithms, entropy salt, etc), that a given system could in theory produce the same output every time as logically as any other number, because the odds of it ever arriving at that one are both exactly the same as every other number it could produce, and therefore just as likely.
So on said system I could request a purely random number between 1 and 10 and it would consistently produce 5, not in error, so if asked to repeat this ten times  5,5,5,5,5,5,5,5,5,5 or 1,2,3,4,5,6,7,8,9,10 is just as likely as any other outcome.
If this is not correct please correct me.
If so, does this theory or law have a name?
 A: The feeling your children have is a common one known as the gambler's fallacy. Put aside your discussions regarding algorithms and salts, and just think of a fair coin. Since every coin toss is independent, it is just as statistically likely to flip and get 10 heads in a row as it is to get HTTHTHHTHT, even though the latter feels "more random".
I remember hearing something about this, I believe from Radiolab's program "Stochasticity", in which two people were given papers. One of the papers had a list of random numbers on it (taken from atmospheric noise or whatever other sufficiently random device you may like to consider), and the other one just had the words "make up random numbers". The people proceeded to write out numbers on the blackboard. When an average observer was asked which of the two blackboards was a list of random numbers, they often picked the wrong one. The reason most of them cited was that the other board (the actually random one) had too many strings of the same number, i.e. 22222 or the like.
The program also interviewed someone who audited taxes, and this person (obviously familiar with the notion) easily picked the truly random board. As it turns out, this is a common rule of thumb when verifying tax returns - people faking the numbers shy away from putting strings of numbers together.
A: You're partially right, but it's worth understanding why we talk about 5,5,5,5,5,5,5,5,5,5 as not being a random outcome.  You mentioned entropy in passing, and that's at the root of the matter: specifically, the so-called algorithmic entropy (or information content, or Kolmogorov complexity, or... - it goes by a lot of names!) of the sequence.
In short, it's true that your process is equally likely to produce 5, 5, 5, 5, 5, 5, 5, 5, 5, 5 as it is to produce 9, 4, 3, 3, 6, 2, 10, 7, 5, 9 - but there are a lot more sequences that 'look like' the latter than the former.  For instance, there are 10! sequences that consist of the numbers 1..10 in some order, but only 10 sequences that consist of 10 repetitions of the same number, so you can say that you're thousands of times more likely to get some permutation of 1..10 - that is, to get each number once - than to get one number ten times.  But the permutations are still a small fraction of the $10^{10}$ different sequences, so you can say that the odds of hitting any permutation are almost vanishingly small; etc.
In general, mathematics formalizes this principle by the aforementioned notion of information content: informally, the notion is 'how easy is it to distinguish this sequence from other sequences?'  The 'all 5s' sequence is easy to pick out of a crowd, whereas the 9, 4, 3, 3, ... sequence is a lot more complicated - it has essentially no description that's shorter than just giving the sequence.  We can formalize the idea by asking for the smallest program that outputs the given sequence; by this metric, it turns out that almost all sequences are random in the formal sense that any program outputting the sequence must be roughly as long as the sequence itself, and so results like 'all 5s' are so rare that they deserve scrutiny even if they're exactly as likely as any other specific sequence.
A: That is correct; in principle a truly random number generator could generate the same number a lot of times. I presume that you have rolled a 6 with a die consecutively numerous times - this does generally not influence your faith in the die being unbiased.
It is the same with random number generators. The principle that is applied is that an outcome (in this case, a random number) does not depend on earlier outcomes, like having rolled a 6 with a die does not influence the chance that the next roll will be a 6 as well.
This is an important principle in cryptography and probability theory; it is related to the concepts of prior and posterior distribution, which have widespread application, perhaps the most in Bayesian statistics.
