Song Property Analyzer beaTlets

As you know, beaTunes has the ability to analyze music files at the audio signal level. Features one might want to extract from the signal are e.g. key, beats per minute (bpm) or time signature (4/4, 3/4, ...). For this kind of analysis, beaTunes uses Jipes. Jipes is an open source digtal signal processing (DSP) library that was created in the process of improving beaTunes. It lets you define a pipeline of processors that transform the signal until you obtain the desired feature. For examples on how to use Jipes, please refer to its documentation. BTW, contrary to Jipes, beaTunes uses native FFT implementations on both macOS and Windows.

While beaTunes can't write your Jipes DSP code for you, it makes it extraordinarily easy to integrate. You just need to implement the com.tagtraum.audiokern.SongPropertyAnalyzer interface, which basically serves as a factory for your pipeline and tells beaTunes which song property to write the result to.

The pipeline itself can rely on the pumped AudioBuffers having a sample rate of 44.1kHz, two channels, being signed, and 16 bits/sample (i.e. values are floats, but in the range of shorts). To map the signal to a value range between -1 and 1, you might want to use new Mapping<AudioBuffer>(AudioBufferFunctions.createMapFunction(MapFunctions.createShortToOneNormalization())).

To make sure that beaTunes picks the correct result from the pipeline, you must give the processor that delivers the feature, an id that is identical to the property name. Typically the last processor in the pipeline is the one that computes the feature. The property name also makes sure that beaTunes knows how to display your analyzer in the beaTunes analysis options dialog.

Because feature extraction is nontrivial, the following examples don't actually compute anything. They are merely meant as illustrations for how to integrate your own pipeline.

# Jython

                        from com.tagtraum.audiokern import AudioClip
from com.tagtraum.audiokern import SongPropertyAnalyzer
from com.tagtraum.jipes.audio import Mono

# Simple song property analyzer.
class KeyAnalyzer(SongPropertyAnalyzer):

    # Name of this analyzer. Please include a version number.
    def getName(self):
        return "Jython KeyAnalyzer 1.0.0"

    # How much audio do we need?
    def getRequiredClip(self, audioFileFormat):
        # AudioClip takes start and stop times in ms
        return AudioClip(0, 120000)

    # Create a new pipeline. See Jipes for details.
    def createPipeline(self):
        # Mono does not compute a key. It's merely a stand-in for a real
        # com.tagtraum.jipes.SignalPipeline
        # The id of the processor that computes the key, must be "key".
        return Mono()

    # The property you want to change.
    # This corresponds to com.tagtraum.audiokern.AudioSong properties.
    def getPropertyName(self):
        return "key"

// Groovy

                        import com.tagtraum.audiokern.*
import com.tagtraum.jipes.*
import com.tagtraum.jipes.audio.*
import javax.sound.sampled.AudioFileFormat

// Simple song property analyzer.
class KeyAnalyzer implements SongPropertyAnalyzer {

    // Name of this analyzer. Please include a version number.
    def String getName() {
        return "Groovy KeyAnalyzer 1.0.0"
    }

    // How much audio do we need?
    def AudioClip getRequiredClip(AudioFileFormat audioFileFormat) {
        // AudioClip takes start and stop times in ms
        return new AudioClip(0, 120000)
    }

    // Create a new pipeline. See Jipes for details.
    def SignalProcessor createPipeline() {
        // Mono does not compute a key. It's merely a stand-in for a real
        // com.tagtraum.jipes.SignalPipeline
        // The id of the processor that computes the key, must be "key".
        return new Mono()
    }

    // The property you want to change.
    // This corresponds to com.tagtraum.audiokern.AudioSong properties.
    def String getPropertyName() {
        "key"
    }
}

# JRuby

                        require 'java'

java_import com.tagtraum.jipes.audio.Mono
java_import com.tagtraum.audiokern.AudioClip

# Simple song property analyzer.
class KeyAnalyzer

    # Implement SongPropertyAnalyzer interface
    include Java::com.tagtraum.audiokern.SongPropertyAnalyzer

    # Name of this analyzer. Please include a version number.
    def getName()
        return "JRuby KeyAnalyzer 1.0.0"
    end

    # How much audio do we need?
    def getRequiredClip(audioFileFormat)
        # AudioClip takes start and stop times in ms
        return AudioClip.new(0, 120000)
    end

    # Create a new pipeline. See Jipes for details.
    def createPipeline()
        # Mono does not compute a key. It's merely a stand-in for a real
        # com.tagtraum.jipes.SignalPipeline
        # The id of the processor that computes the key, must be "key".
        return Mono.new()
    end

    # The property you want to change.
    # This corresponds to com.tagtraum.audiokern.AudioSong properties.
    def getPropertyName()
        return "key"
    end
end

// JavaScript

                        /*
 * These type vars basically act as imports for Java classes.
 */
var SongPropertyAnalyzer = Java.type("com.tagtraum.audiokern.SongPropertyAnalyzer");
var AudioClip = Java.type("com.tagtraum.audiokern.AudioClip");
var Mono = Java.type("com.tagtraum.jipes.audio.Mono");

var beatlet = new SongPropertyAnalyzer() {

    /*
     * Unique id of this analyzer.
     * This is essential for SongPropertyAnalyzers
     * written in Javascript.
     */
    getId: function() {
        return "JavascriptKeyAnalyzer1.0.0";
    },

    /* Name of this analyzer. Please include a version number. */
    getName: function() {
        return "Javascript KeyAnalyzer 1.0.0";
    },

    /* How much audio do we need? */
    getRequiredClip: function(audioFileFormat) {
        // AudioClip takes start and stop times in ms
        return new AudioClip(0, 120000);
    },

    /* Create a new pipeline. See Jipes for details. */
    createPipeline: function() {
        // Mono does not compute a key. It's merely a stand-in for a real
        // com.tagtraum.jipes.SignalPipeline
        // The id of the processor that computes the key, must be "key".
        return new Mono();
    },

    /*
     * The property you want to change.
     * This corresponds to com.tagtraum.audiokern.AudioSong properties.
     */
    getPropertyName: function() {
        return "key";
    }
}

// Put "beatlet" into the last line, so that it is returned to beaTunes
// when this script is eval'd.
beatlet;

For writing a custom key renderer, please check out the key text renderer sample.

Other beaTlet samples:

All sample beaTlets are also on GitHub .