| mp4-sa-> the mp4-sa book-> special topics->audiounits |
Sections
|
Naming ConventionsParameter Sliders  Parameter Text Menus  Factory Presets  AU Type  AU Subtype AU Manu  AU Filename  AU Bundle Manu  AU Custom Views  Standard Namescpuload  |
IntroductionPlug-ins are small programs that can be run within larger host applications. In the audio world, host programs are often digital audio workstations, and plug-ins often act as in-line effects (doing tasks like equalization) or as virtual musical instruments (accepting MIDI input and generating audio output, in the way a hardware synthesizer does). By supporting plug-ins, a host developer lets outside developers write programs that run inside the host. In this chapter, we describe how to write audio plug-ins in SAOL. At this time, sfront supports a single plug-in standard -- the AudioUnit standard on Mac OS X. This format is supported by many OS X applications, including GarageBand, Logic, Ableton Live, Digital Performer, and Final Cut Pro. SAOL plug-ins are known to work under Lion (10.7) and also under Tiger (10.4). Compatibility reports for Leopard and Snow Leopard would be appreciated. Plug-in support in sfront is a work in progress, and in this chapter we note the current limitations of writing plug-ins in SAOL. Most of these limits are not intrinsic to using SAOL for plug-ins, but rather reflect the current "gaps" in sfront support. We anticipate that most of these gaps will be filled as sfront AudioUnit support matures. |
|
AudioUnits DemystifiedNewcomers to plug-ins are often surprised at how much extra programming effort a plug-in involves, compared to a stand-alone command-line program. The difference has little to do with the the audio signal-processing code for the program. Below, we list the major reasons why AudioUnit programming is complicated, and why writing a plug-in in SAOL reduces the complexity. |
|
DiscoveryA host program needs to discover the plug-ins that are installed on a computer, in order to support the user interfaces that let users select plug-ins to add to a project. The AudioUnit discovery process requires that users install AudioUnits in special directories on the disk file system (listed on the right panel). Host programs examine these directories to build the user-interface menus that display plug-ins to the user. AudioUnit plug-ins appear in the Finder as individual files, but are actually a special type of directory tree called a bundle. The right panel shows a typical directory structure for an AudioUnit. A special file, called a resource file (.rsrc), codes discovery information about an AudioUnit, including the data the host uses to build its user-interface menus for plug-in selection. When building an AudioUnit from a SAOL file, sfront creates the resource file automatically. The example Makefiles shipped with the sfront distribution create the correct bundle directory structure, and place the resource file in the correct place in the bundle. To build an example AudioUnit, type make install. This command builds the AudioUnit, copies it into a discovery directory, and launches an AU host to begin testing. |
Discovery Directories
All users: /Library/Audio/Plug-Ins/Components/
A specific user: ~/Library/Audio/Plug-Ins/Components/
AudioUnit Bundle StructureFor AudioUnit hiss: % ls hiss.component/ % cd hiss.component % ls Contents/ % cd Contents % ls Info.Plist MacOS/ PkgInfo Resources/ % cd MacOS % ls hiss* % cd ../Resources % ls hiss.rsrc |
Dynamic LoadingThe C code of an AudioUnit does not include a  main() function. As a consequence, an AudioUnit cannot execute as a stand-alone program. Instead of a  main() function, an AudioUnit defines an entry function that implements a standardized interface between a host and the plug-in (called a Component interface). The name of a plug-in's entry function is encoded in its resource file, along with the name of plug-in binary file. When the user chooses to add a plug-in to a project in a host, the host dynamically maps the plug-in binary into the host's address space, and then calls the plug-in's entry function. The host uses a standard interface to communicate with the plug-in through the entry function. The information in the resource file enables this bootstrap process. The sa.c file created by sfront defines an AudioUnit entry function. The entry function bridges the gap between the AudioUnit standard and the MPEG 4 Structured Audio standard. The resource file that sfront creates embeds the names of the entry function and the AudioUnit binary file, and the Makefiles shipped in the sfront AudioUnit example projects place the binary file in the correct location in the bundle directory tree. |
|
Multiple InstantiationA host may be running several copies (instances) of an AudioUnit plug-in at the same time. Each instance acts as an independent entity. However, only one binary image of the AudioUnit plug-in is mapped into the host's address space. You may be wondering, how do the AudioUnit instances maintain independent state while sharing a common binary? To solve this problem, AudioUnits use an "instance factory" model. An AudioUnit entry function accepts host requests to create independent instances of the plug-in. The entry function also accepts requests to destroy a plug-in instance when it is no longer needed. As part of the creation process, an entry function dynamically allocates memory to hold an instance's state. The entry function reclaims the state memory when the instance is destroyed. A SAOL program represents a single instance of the plug-in. The sa.c file created by sfront uses this SAOL program as a template to create independent instance copies on demand. Sfront detects SAOL global variables that are accessed in a read-only fashion, and shares those variables among all instances. This optimization is particularly helpful for plug-ins that use large sample wavetables. |
|
User InterfaceIn addition to processing audio, an AudioUnit plug-in may present a control interface to the user inside the host. The host provides a configuration button for each plug-in instance; the user the button to bring up the control interface window. For example, the interface of a high-pass plug-in may include an on-screen slider to vary the filter cut-off frequency. A control screen for this plug-in is shown below.
An AudioUnit defines its user interface by declaring floating-point parameters during initialization. For each parameter (for example, the cut-off frequency parameter of a high-pass filter plug-in), the plug-in tells the host the parameter name ("Cut-Off"), its units ("Hz"), its preferred user interface ("a slider with a logarithmic taper"), and the parameter's minimum ("1 Hz"), maximum ("300 Hz") and default values ("80 Hz"). A host creates a simple (a.k.a generic) user interface for a plug-in, by drawing a user-interface control on the screen for each parameter. The host sends an updated parameter value to the plug-in whenever the user moves a slider. Parameters are also the mechanism AudioUnits user for persistent state -- when a host saves a user project, it saves the current parameter values for each plug-in. SAOL programmers specify the parameters for a plug-in's user interface by declaring ksig variables in the global block that follow a special naming convention. Ancillary variable declarations define the user-interface appearance of each parameter. The naming conventions for these SAOL variables are defined on the right panel. On each k-cycle, the values of SAOL global ksig variables associated with AudioUnit user-interface parameters are updated to reflect user activity in the host. By importing these global variables, SAOL instrs can change their behavior based on real-time user input. The variable updates follow the same semantics as SASL updates of global variables. If SAOL program code updates a variable that is associated with a AudioUnit user-interface parameter, the new data appears in the AudioUnit user-interface screen. The naming convention supports display user-interface elements, so that the user-interface can display parameter values (without offering a way for the user to change the parameter values). |
Parameter Naming ConventionsAudioUnit parameters are SAOL ksig scalar variables defined in the global block. Variable must begin with aup_, followed by the parameter name. In the user interface, underscores in the name are presented as spaces, and case is preserved. The name must not use the strings _factory, _idx, _unit, _slider, _menu, _checkbox, and _pinfo. SAOL programs read the aup_name variable to learn the current value of a parameter.
The user-interface control for a parameter aup_name may be
specified by declaring a ksig scalar variable with a particular
naming convention. This variable exists to code information in its
name -- the run-time value of the variable has no special
significance to the sfront engine. The examples below define the
naming convention:
In a similar fashion, the unit label that appears next to a slider
may be specified with variables of the form:
The range and default value for a parameter aup_name is defined by a SAOL table aup_name_pinfo. The three elements of this table specify the minimum, default, and maximum values of the parameter.
The parameter variable in the example control window is defined as:
|
Factory PresetsAn AudioUnit plug-in developer may define factory presets. Factory presets are sets of parameter value that produce a useful behavior. For example, a high-pass filter plug-in may define a factory preset "Vocal Rumble", whose cut-off value is useful for filtering out the rumble picked up by microphones stands. SAOL programs may define a factory preset by declaring a table in the global block that follows a specialized naming convention. The naming convention is shown on the right panel. By enabling a special debug mode, SAOL developers can instruct a plug-in instance running in a host to print out a factory preset table of the current plug-in state to a log file. We describe this debug mode later in this chapter. |
Factory Preset Naming ConventionsAudioUnit factory presets are defined as tables in the global block. Preset tables must begin with the prefix aup_factory_, followed by the preset name. In the user interface, underscores are presented as spaces, and case is preserved. A preset table has a length equal to the number of AudioUnit parameters. Each table entry corresponds to a parameter, following the order of parameter declaration in the global block. A parameter's value for a factory preset is the computed value of its table entry in the factory preset table.
Example factory preset declarations:
global {
// Parameters shown in AudioUnit user interface
ksig aup_Attack;
ksig aup_Release;
// Factory preset tables Attack Release
table aup_factory_Quick(data, 2, 0.005, 0.025);
table aup_factory_Normal(data, 2, 0.05, 0.075);
table aup_factory_Slow(data, 2, 0.200, 0.200);
}
|
SAOL Plug-insThe AudioUnit plug-in standard is deep and wide. At present, plug-ins written in SAOL support a subset of the full AudioUnit specification. Below, we describe the current status of SAOL plug-ins. |
|
Plug-In TypesThe AudioUnit standard defines a family of AudioUnit types. The three most popular types are currently supported by SAOL: MusicDevice, Effect, and MusicEffect. The right panel shows the sfront command-line options for each type. MusicDevice plug-ins are software synthesizers: they accept MIDI input (including note events) and generate audio output. Effect plug-ins are audio signal processors: they accept audio input and produce audio output. MusicEffects are similar Effects, but also accept a MIDI input stream. MusicEffects are useful for creating software synthesizers that require audio input, such as envelope followers and vocoders. The resource (.rsrc) file encodes an AudioUnit's identity using three fields: the Type, Sub-Type, and Manufacturer fields. Each field holds a 4-character constant. Sfront automatically encodes the correct Type constant in the resource file (aumu for MusicDevice, aufx for Effect, aumf for MusicEffect). The Manufacturer and Sub-Type fields form a unique identifier of a plug-in of a particular type. The Manufacturer field identifies the author of the plug-in, and the Sub-Type field identifies the particular plug-in product created by the author. The right panel shows sfront command-line options to set the Manufacturer and Sub-type fields, along with related options to set the plug-in bundle name (as seen in the Finder), and to code descriptive information about the plug-in that is shown in the user interface of an AudioUnit host. AudioUnit Bundle IDAudioUnits are a type of bundle. Bundles include a file Contents/Info.Plist that describes the bundle. The CFBundleIdentifier property in this file acts as a bundle identification string. Sfront automatically creates the CFBundleIdentifier string, and places it in the Info.Plist file. This string begins with a sub-string defining the manufacturer of the AudioUnit, using an inverted version of a DNS name associated with the manufacturer. For example, we use edu.berkeley.eecs for the example plug-ins that ship with sfront. The right panel shows the command-line options for setting the manufacturer sub-string for use with CFBundleIdentifier. |
Type SpecificationA MusicDevice AudioUnit (type aumu) is specified by the sfront command line options: -aout audiounit -cin aucontrolm An Effect AudioUnit (type aufx) is specified by the sfront command line options: -aout audiounit -ain audiounit -cin aucontrol A MusicEffect AudioUnit (type aumf) is specified by the sfront command line options: -aout audiounit -ain audiounit -cin aucontrolm In each case, -aout audiounit_debug may be substituted for -aout audiounit to use a debugging version of the audiounit drivers. All other combinations of (-ain, -aout, -cin) that include an AudioUnit driver will result in an sfront compile-time error. Sub-Type and Manufacturer OptionsSfront command-line options set the AudioUnit subtype and manufacturer fields. Each option flag must be followed by a four-character literal to define the field. Examples below: -au_component_subtype Hiss -au_component_manu ucBe The name for the AudioUnit bundle (as it appears in the Finder) may also be specified by an sfront command-line option. Spaces should not appear in this name. An example below: -au_filesystem_name hiss Finally, command-line options may be used to set the AudioUnit name and manufacturer, as it is presented in the user interface. Examples below: -au_ui_name "Inline Noise Generator" -au_ui_manu "John Lazzaro, UCB EECS" Bundle Identifier OptionsSfront command-line options determine the bundle identifier string that appears in the Info.Plist file. Example below: -au_manu_url edu.berkeley.eecs -au_filesystem_name hiss creates the CFBundleIdentifier string: edu.berkeley.eecs.audiounit.hiss |
Sample RatesIn most cases, users configure an audio application to run at a particular audio sample rate. Users expect plug-ins to run at this sample rate. However, SAOL programs typically set a constant audio sample rate in the global block, using the srate parameter. SAOL plug-ins handle this situation in the following way. SAOL programs are free to set the srate parameter. and the sa.c sends a request to the host to use this sample rate during initialization. However, the host may override the request, and force the plug-in to use a different sample rate. In all cases, the SAOL standard name s_rate codes the actual audio sample rate. SAOL programs should use s_rate in all expressions, rather than the constant value for the parameter srate set in the global block. The SAOL language standard requires that the krate of an orchestra evenly divide the srate. If a plug-in host changes the audio sample rate of a SAOL program, the krate may also change slightly, so that it still evenly divides the srate. In all cases, the SAOL standard name k_rate will code the actual control rate. SAOL programs should use k_rate in all expressions, rather than hard-code the constant value for the parameter krate set in the global block. |
K-rate CaveatsIn the current sfront implementation, there are a few places where k-rates that change during run-time (due to an AudioUnit host changing the audio sampling rate) are not handled gracefully. These errata (and work-arounds) are listed below. (1) fft()/ifft()If len is not specified in the core opcodes, len defaults to the smallest power of two that is greater than s_rate/k_rate. At present, this computation is done at sfront compile-time, and thus will not reflect krate changes during run-time. As a work-around, specify len (or other parameters like size) explicitly, using the standard names s_rate and k_rate to compute the desired length. (2) SASL and MIDI file playbackIf your AudioUnit specifies events in a SASL or MIDI file for playback, the temporal sequencing of these events will be hard-coded in the sa.c file, using the compile-time k_rate as a time base. If the k-rate changes during run-time, playback will occur at the wrong tempo. The easiest workaround is to set a krate that evenly divides the popular audio sampling rates (96000, 88200, 48000, and 44100 Hz). The greatest common divisor is 300 Hz. |
Audio BusesAudioUnits support a versatile audio bus model. A plug-in may have an arbitrary number of input and output audio buses, and each bus may have an arbitrary number of channels. The SAOL audio bus model, at present, is more restrictive, as we describe below. However, we intend to support a more versatile bus model in future sfront releases. During initialization, a SAOL plug-in informs the AU host that the plug-in can deliver mono or stereo audio output on a single bus. An Effect or MusicEffect plug-in also informs the host that it can receive input on a single mono or stereo bus. The host is free to choose a mono or a stereo bus for both input and output. However, the plug-in's SAOL code is not informed of the bus sizes selected by the host. Instead, the SAOL code is written to support a static output_bus width and (for Effect and MusicEffect plug-ins) a static input_bus width. The global parameters outchannels and inchannels signal this choice (legal values are "1" and "2"). If an AU host selects a mono input bus for a SAOL plug-in that uses a stereo bus, the sa.c replicates the mono signal on the left and right channels of the SAOL input_bus. Complementary conversions are performed for the stereo-to-mono input case, and for channel-width mismatches for audio output. Note that the "single-bus" restriction means that at present, SAOL does not support plug-ins that require a side-chain input bus. |
|
User Interface ViewsThe AudioUnit specification lets a plug-in replace the generic user-interface controls (described earlier in the chapter) with a custom user-interface that the plug-in draws for itself. Most commercial plug-ins ship with a custom interface. SAOL plug-ins are capable of using a custom user-interface view, as we describe in the lpf example later in this chapter. However, at present, sfront does not create a custom user-interface view from the SAOL program structure (we intend to do so in a later release). At present, programmers write the user-interface view for a SAOL plug-in in Objective-C, using the Cocoa framework. |
|
Unsupported FeaturesIn addition to the limitations described above, many other AudioUnit specification features are not supported by SAOL plug-ins at present. For example, at present there is no way for a SAOL plug-in to sense the current tempo of an AU host sequencer. Thus, delay plug-ins that synchronize to tempo are not possible to write in SAOL. As a second example, the AudioUnit specification provides several mechanisms to ensure that plug-ins with long "decay times" (such as reverbs) clear their internal state in response to sequencer transport controls. At present, these decay-time features are not supported by SAOL plug-ins. We anticipate that these two unsupported features, and many others of a similar nature, will be supported in SAOL plug-ins in the future. In the sections that follow, we show examples of SAOL plug-ins. The examples are included in the sfront distribution examples directory, and are intended to serve as starting points for your own projects. The examples are known to work on several AudioUnit hosts: GarageBand, Ableton Live, AU Lab (a host distributed with Apple's Developer Tools) and the command-line validation host auval. |
|
sin: A Softsynth ExampleThe right panel shows the SAOL code for sin, a simple sine-wave virtual instrument that ships with the sfront distribution, in the directory sfront/examples/rtime/au/sin. Typing make install in this directory builds the AudioUnit MusicDevice component sin.component. The sin plug-in acts as as MIDI-controlled polyphonic music synthesizer. It should appear in the AU-host menu that lets the user choose MIDI instruments. The AudioUnit responds to MIDI note commands on any MIDI channel, and generates audio output in response. Global BlockThe global block begins by setting the default srate and krate values. However, all references to these values in sin are made via the s_rate and k_rate standard names. This practice ensures that sin uses the actual k-rate and s-rate of the plug-in (recall that an AudioUnit host may change the audio sampling rate of a plug-in instance, and this change in the s-rate may also force a change in the k-rate). The global block also sets outchannels to configure the output_bus to be mono. If an AudioUnit host inserts sin into a stereo bus, the output_bus value will be copied by the sa.c driver code to both AudioUnit stereo channels. The rest of the global block defines three AudioUnit parameters (Attack and Release, to shape the envelope of the note waveforms, and Notes On, to display whether notes are currently sounding) and several factory presets with different envelope shapes (Quick, Normal, and Slow). The declarations follow the naming conventions defined earlier in this chapter (click for parameter and factory preset conventions). To keep the example simple, no send statements appear in the global block, and thus SAOL instrs launched by MIDI Note commands sum audio output to the output_bus. In practice, most MusicDevice AudioUnits written in SAOL would use send and route statements to post-process this output with effect SAOL instrs, as shown in this example in the tutorial chapter of this book. |
//
// AudioUnit MusicDevice (aumu) example
// Sine wave synth with variable attack and release
//
global {
srate 44100; // AU host may change this value
krate 1050;
outchannels 1;
// Parameters shown in AudioUnit User Interface
ksig aup_Attack;
ksig aup_Release;
ksig aup_Notes_On;
// Sets the user-interface widget for each parameter
ksig aup_Attack_slider_log;
ksig aup_Release_slider_log;
ksig aup_Notes_On_display_checkbox;
// Sets the unit to display next to the slider
ksig aup_Attack_unit_s;
ksig aup_Release_unit_s;
// Parameter info min default max
table aup_Attack_pinfo(data, 3, 0.001, 0.01, 0.25);
table aup_Release_pinfo(data, 3, 0.001, 0.05, 0.25);
table aup_Notes_On_pinfo(data, 3, 0, 0, 1);
// Factory Preset Tables Attack Release NoteOn
table aup_factory_Quick(data, 3, 0.005, 0.025, 0);
table aup_factory_Normal(data, 3, 0.05, 0.075, 0);
table aup_factory_Slow(data, 3, 0.200, 0.200, 0);
// Note: Attack appears first in the tables because
// it is declared first in the global block.
ksig note_count; // number of notes playing
}
|
The tone instrThe rest of the sin example program defines the tone MIDI instr, shown on the right panel. An instance of tone is created for each MIDI NoteOn received from the AudioUnit host. In most respects, AudioUnit MIDI instrs follow the semantics defined in the MIDI control chapter of the book. However, the sin example uses aspects of sfront initialization behavior simplify AudioUnit power-up behavior, in the manner we now describe. As shown on the right panel, the tone header specifies preset 0, indicating that a tone instance should be created for NoteOn commands on all MIDI channels whose most recent MIDI Program Change set the program number to 0. The reader may wonder, how does the AudioUnit behave if the user never sends a MIDI Program Change command to the plug-in? The answer is as follows. During compilation, sfront sorts all MIDI instrs in order of ascending preset tag number. Then, sfront initializes MIDI channel 0 to use the first instr in this sorted list, MIDI channel 1 to use the second, etc. If there are more channels that list elements, the list is traversed several times. Thus, for the sin program example, all MIDI channels execute a MIDI Program Change 0 command on power-up, and all Note commands will create a tone instance. By default, tone will remain bound to all MIDI channels for the lifetime of the AudioUnit, as the sfront run-time engine ignores MIDI Program Change commands whose preset values do not correspond to a SAOL instr. The sfront option -null-program changes this behavior, so that MIDI Program Changes to preset numbers without corresponding SAOL instrs will silence a MIDI channel. tone program codeThe tone instr imports the SAOL global variables associated with AudioUnit user-interface sliders, and uses these variables to initialize the ksig variables that generate the attack envelope phase. The initialization occurs at instance-relative time 0, inside the if (itime == 0) clause). The rest of the k-rate code detects the MIDI NoteOff (by sensing the released standard name), extend()s the lifetime of the instance to execute the release envelope, and reconfigures the envelope generation variables to create the release envelope phase. The a-rate code generates the sine waveform using a simple trigonometric algorithm, shapes the sine wave with the envelope waveform, and uses an output statement to sum the audio sample onto the output_bus. |
|