While messing around with open harmonics on a guitar string I came to the same question about diatonic scaling after noticing that the natural harmonics that sound over the 5th, 7th and 12th frets also happen to mark fretted intervals that sound the perfect 4th, and 5th as well as the octave, and these nodes also represent perfect divisions of the string into 3 and 4 segments.
Any ancient budding musical physicist would easily find that not only is there a relationship between string length and pitch, but there is also a harmonious relationship between proportional string lengths. Experimenting with lengths of hollow pipe, such as with pan pipes, would produce the same kind of results, a discovery of harmony within perfect ratios.
I think this can explain why diatonic and pentatonic scale systems seem to have been discovered independently by multiple ancient cultures. Its just a natural 'goto' if you are looking for harmony or to play with others.
The earliest attempts at polyphony seem to be to drone on a single note while a lead melody articulates over the top, we see this with Australian aborigines and their didgeridoo, Ancient Greek aulos, Roman bag pipes, Gregorian chant. The idea came up through the ages.
Within the artwork of our ancient civilizations we find many images of musicians playing in groups, multiple musicians. It seems with music comes a desire to connect and harmonize with others. Again the natural tendency will lean towards a more simple octave division if you want more than two people to be able to hit any 3 notes and harmonize.
You can go as far back as 2500BC where we see Mesopotamians playing poly stringed instruments, or ancient Egyptian pottery showing the hand positions of harp players, with hands stretched out to be able to pluck multiple notes at once. These people were playing chords, and harmonizing in groups, they were not just monotonic. The diatonic and pentatonic unlocked polyphonic harmony for them, without this you end up locked to droning instruments with atonal vocal blither over the top, have a listen to an aborigine jamboree some time. And if not mentioned somewhere already, you can even go back to 7000-5000BC to Jiahu in Asia where we find pentatonic and diatonic flutes.
So one of the first really interesting things you discover when you get a stringed instrument is how haunting a sound is created when you strike that central harmonic, as it brings out two overtones in unison, with added harmonic effects caused by ring modulation over the length of the string, a subtle variation that sounds a bit gong like. It is not long before you discover there are a few more nodal points like this, they are not as pronounced but they bring out different pitched overtones and their pitch is in near perfect harmony with the tonic vibration.
Exploring a guitar string from the open E, the harmonic nodes reveal the position of where the A and B fret would reside, and when you sound these two notes together E&A or E&B you get those beautiful harmonic 4th and 5th dyads. Our ancients would surely say, 'wow that sounds really nice', we pretty much all do.
But because these natural harmonics are the result of the divided ratio of the string, and it doesn't matter what the length of the string, it would then follow that more natural harmonics could be discovered if you then used the harmonics of the fretted A and fretted B notes, to see what new potential harmony is revealed.
Sure enough for the fretted B, you discover the high B of the string at the mid point, a node at E @12th fret and F# @ the 14th.
From the A we discover the D @10th fret and again the E @12
Now we have found 5 notes that we know are in some kind of strong harmonic relationship to each other.
E F# A B D
But hello ... we have a pentatonic scale.
Note we could also use these harmonics as a tuning system, when we work out how to modify tension with pegs, they surely did this in Mesopotamia, we've found found their texts where they talk about doing this.
but if we now take this idea one step further.. and apply the same harmonic search idea to our newly discovered D and F# we get.
From the D we discover the G
From the F# we discover the C#
Now we have a diatonic 7 note scale
E F# G A B C# D
While these discovered notes are rather high in pitch relative to the original tonic note, it would not be a difficult thing to apply a 'rule of octaves' which states any length that is exactly half the length, or exactly twice the length will reveal some kind of perfect mathematical 2:1 harmony.
From here we are on a path of discovery of the circle of 5th's and if you keep going with this exploration you will discover the full 12 notes of the chromatic scale. You might stop there, or you might find that there are harmonic anomalies and start adding micro-tones to deal with say .. 'Pythagorean wolf notes'.
Given how natural and fundamental these harmonic relationships are, it is no surprise that the same scaling keeps coming up all over the place, and could be discovered and rediscovered throughout history and within the span of any averagely inquisitive individual's lifetime, especially if they had some intent to create harmony with others.
I think because of how fundamental these prime relationships are, that even if we run into alien species halfway across the galaxy they are very likely to also have music based on diatonic scales and triads. Or possibly if their senses allow, some other kind of diatonic vibrational system of harmony within the spectrum of forces and fields like magnetic fields for example.
And here I will digress and imagine where else these natural harmonic nodal points could be applied.
Like it would be interesting to see what happens say if you create a vibrator / massager that operates with 3 vibrators that are in proportional harmonic relationship to each other and apply that to your skin and muscles, or even directly to your nervous system say via pulsating electric currents. Would we get the same pleasurable sensations of harmony as we do from a natural 1 3 5 triad? I bet we would.
And what happens if we try to tune an engine like this? With 3 rotors or pistons say that are spinning at these ratios? Do we get a more balanced and harmonious engine that in some way is able to put out more power or last longer because of the harmonic balance and possible efficiency that comes with it? Or does the opposite happen, with wild systemic harmonic build up that tears the system apart? I am betting there is some crazy power to be disovered when you get the vibrational resonance right between the vibrations of exhaust and torque conversion. Actually I think electric engines are built on a 3:4 ratio between 3 coils on the armature and a surrounding static magnetic field that alternates between 4 stages of negative and positive. Something tells me there are more applications to this that are as yet unexplored and very high powered engines are possible.
Like for example if you have harmonics ringing across a length of axle and you know there is going to be a harmonic nodal point right at the center point of the axle, this would surely be the best place to position a drive axle for the least disturbance and vibration, not as in the vast majority of cases where drive gears are typically off to the side somewhere and surely prone to greater vibration. Quite often when you observe these systems the chains are vibrating wildly. Imagine if you are dealing with a really lightweight future vehicle with a flexible carbon fiber axle, or a high speed go kart which typically have hollow flexing axles, you might be able to use the axle itself for suspension if the gears are positioned within the harmonic nodes.
It would be exciting to see a machine where it's parts are harmonically balanced around triads and diatonics, like a hybrid rotary combustion / electric engine that sings a major chord when it revs up, drops to minor when you shift to 2nd gear and grinds down with a flat 5 when you compression brake it. An engine singing a C Major triad as you rev it up would be so incredibly awe inspiring.
And what about light, with it's visible spectrum between 400nm and 800nm. I bet there exists the same diatonic scale and that a spectrum where the wavelengths are divided with the diatonic ratios will produce a set of colors that work together with the same visual harmony as musical harmony and the intervals between the hues of that scale will resemble the same intervals we see on the guitar string.. So a color triad of a 135 should work very well. If the scale is not exact, it will be very close. The actual colors themselves will concentrate more strongly and distinctly within the pockets of the diatonic intervals, giving you 7 strong and distinct hues with the white and black on the octaves, and also giving intervals between the colors that represent whole tone and semi tone jumps, such is the case between green cyan and violet, a semitone step into cyan with two dominant hues on either side of it. The colors found near the strong harmonics we see near the 5th and 7th frets, the 1/3 and 1/4 divisions will be very dominant and cover a wider bandwidth as they will overpower their neighboring harmonics, I think here we find the color green covering a wide span of the spectrum dominantly, the equivalent to our natural 5th or the dominant. And all that being true, we should also see this same phenomena to some degree within the quantum space of atoms, 7 harmonic nodal points where the energy of matter collects in harmony, for the same reason that we know to the ear as the 4th and 5th taking dominance in harmony, and anything outside of those nodes having a very hard time staying in existence or at least having a hard time getting along with others.
And if you expand that thought out as we did to discover the full 12 note chromatic scale, anyone looking for a way to challenge theories like grand unified or dimensional string theory should quite possibly not be looking for 10 or 11 dimensions but actually 12 dimensions as in 12 nodes on the chromatic scale.
So I think those 7 divisions of an octave are in some way cosmologically standard and there is a good reason different cultures have fallen upon them repeatedly. The diatonic is absolutely optimal.
(I reserve the right to be completely and utterly wrong, and atrociously guilty of plagiarism, please excuse, I'm an old man who has lost his memory of where and how these thoughts came about.)