I'm trying to come up with an algorithm for detecting the notes played in a song. So far i've gotten the "chunks" of data that represent an audio clip. The volume of the chunk, along with the presence of each frequency in that particular chunk (left to right). The number in parenthesis is the increase of vol % over the last 4 chunks.

See here: http://imgur.com/a/xAh3T

The sample above is from a recording i made by stepping the D, G, and then B strings on the guitar. As you can tell, to us humans we can fairly easily see the notes being played in the given data. I am wondering how might be best to algorithmically deduce when a note is played.

A few things i've thought about playing around with...

  • Multiplying a note's presence by the volume or volume slope of a particular chunk. As a note being played is generally associated with an increase in volume. Or at least somehow incorporating the volume slope into my calculations.
  • Averaging the "presence" of a note over a span of chunks, smoothing out the data in a way.
  • Looking at variables such as the standard deviation of the "presence" of a particular note over the period of time it generally takes for a note to start and finish manifesting within the data (as you can see by my data it takes about 5 chunks for the note to fully come into play). When that number increases by a certain SD threshold, that would mean a note was played.
  • Filtering out irrelevant chunks by volume or volume slope.
  • $\begingroup$ How was the presence of the notes in a given timeframe determined? $\endgroup$ – Karlo Sep 1 '17 at 16:16
  • $\begingroup$ This is an extremely tough problem. labrosa.ee.columbia.edu does research on it and related problems and has a lot of interesting information about them. $\endgroup$ – MJD Sep 1 '17 at 16:28
  • 1
    $\begingroup$ @Karlo FFT. I've also considered doing a "presence map" so to speak where you average a note's presence among a span of a couple chunks to smooth the data in a way. $\endgroup$ – Ben Arnao Sep 1 '17 at 16:31

A possibility is to filter the sound with filterbanks corresponding to all A's, A#'s etc. The presence of energy at a certain time point in the filtered version corresponding to a given note, indicates that the note is active at that time.

The algorithm has to be able to distinguish frequencies that are fundamentals (played notes), and overtones of these fundamentals. For this, a model of (quasi)harmonic frequencies can be considered.

Have a look at the literature: chroma pitch filter, automatic pitch detection.

  • $\begingroup$ I keep hearing about overtones (which are multiples of the frequency from my understanding) but they never seem to be relevant in my data for some reason. $\endgroup$ – Ben Arnao Sep 1 '17 at 16:32
  • $\begingroup$ In theory, they are multiples of the fundamental frequency, but they can be inharmonic (such as in the piano). In some cases, they could cause a false detection of a note that is in fact a very present overtone of an other active note. $\endgroup$ – Karlo Sep 1 '17 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.