The MP4-SA Book: Part III/2: MIDI Instrument Control
Description: MIDI Control of SAOL Instruments. Describes how to use a MIDI file or real-time MIDI stream to instantiate SAOL instruments. Describes standard names for MIDI.
Keywords: MP4-SA, MIDI, MIDI file, note on, note off, noteon, noteoff, aftertouch, velocity, channel, extended channel, controller, controller change, preset change, bank select, pitch wheel, tempo, tracks, MIDIbend, MIDIctrl, MIDItouch, preset

### From The MPEG-4 Structured Audio Book by John Lazzaro and John Wawrzynek.

# Part III/2: MIDI Instrument Control

## Sections Introduction. MIDI Basics. Overview of MIDI commands. MIDI Files and SAOL. Trigger times and channels. SAOL Instruments for MIDI. Specializations. Standard Names. Access the MIDI data stream. Streaming and MIDI. MIDI commands without trigger times. | ## Standard Names: MIDIctrl MIDItouch MIDIbend channel preset ## Language Elements: preset

## Introduction MIDI (Musical Instrument Digital Interface) is the dominant score language in electronic music. Most electronic musical instruments have network ports that communicate MIDI commands. Many personal computers also have MIDI network ports. Most musical software applications use the MIDI File Format, a score language based on MIDI. MP4-SA supports the MIDI File Format as an alternative score language to SASL. MP4-SA also supports the inclusion of streaming MIDI commands in an MP4 bitstream. In this chapter, we give a basic introduction to MIDI, and show how SAOL programs interact with MIDI commands. |

## MIDI Basics In this section, we describe MIDI commands and MIDI Files, focusing on features relevant to MP4-SA. MIDI was originally designed to let musicians use a single electronic piano keyboard to play notes on several different electronic musical instruments. In a typical architecture, the piano keyboard sends out control information, in the form of MIDI commands, on a "MIDI Out" network port (a 5-pin DIN connector). This port is wired to the "MIDI In" port of one of the instruments. This instrument regenerates the data on a "MIDI Through" jack, that is connected to another instrument. In this way, several instruments are daisy-chained together, each listening to the same stream of MIDI commands. Most MIDI commands include a channel number field (an integer between 0 and 15). In a typical installation, each instrument is programmed to only respond to commands that have a specific MIDI channel number. #### MIDI Commands See the right panel for the syntax of the MIDI commands discussed in this section. Although these commands are defined as binary objects, in this chapter we discuss MIDI using a symbolic notation. The MIDI commands for starting (NoteOn) and ending (NoteOff) sounds are defined using the vocabulary of the piano keyboard. These commands specify a key number (an integer between 0 and 127, with 60 being Middle C), the velocity that the note was pressed or released, and a channel number. Electronic keyboards often have pressure sensors underneath the keyboard that generate an additional control signal while a note is depressed. Some keyboards have a single sensor for the entire keyboard, while others have a dedicated sensor for each note. The MIDI commands Touch and CTouch send information about the current pressure under a single key or the entire keyboard, respectively. Many electronic keyboards also have a wheel on the left side of the keyboard, with a center detent. Moving the wheel off the center detent acts to smoothly bend the pitch of all depressed notes. The PWheel command sends the current value of the pitch wheel. Electronic musical instruments typically have the ability to play many different types of sounds. MIDI refers to these sound types as presets, numbered between 0 and 127. The MIDI PChange command instructs an instrument on a particular channel to switch to a particular preset. In addition to pitch wheels, electronic piano keyboards often have other attached controllers, such as pedals, secondary wheels, etc. The CChange command lets a keyboard send the value of a controller on a specific MIDI channel. The controller in the CChange command is specified as a number between 0-127, with many numbers linked to specific devices (for example, the piano sustain pedal is controller 64). Other controller numbers are used to extend other MIDI commands. For example, controller 32 extends the PChange command, by specifying the bank of 128 presets that the PChange command accesses. #### MIDI Files As MIDI ports became available for personal computers, sequencer software became available for recording MIDI data from electronic piano keyboards and assembling the data fragments into a complete performance. The MIDI File Format became the standard way to transfer performances between different sequencer packages. The MIDI File Format adds trigger times to the MIDI commands described in the last section. MIDI Files support trigger times with units of beats, with tempo commands to map between beats and seconds. The simplest MIDI File type, Format 0, stores all note data in a single track, which consists of a list of MIDI commands with time triggers. This format limits the performance to use 16 MIDI channels. Format 1 MIDI files store note data on multiple tracks, with each track having a distinct 16 MIDI channels available. This format supports an arbitrary number of channels. | ## MIDI Command Syntax `
    NoteOn Ch Note Velocity

    Ch: 0-15. The channel that should
    play the note.

    Note: 0-127. The piano keyboard
    note to play. 60 is middle C.

    Velocity: 0-127. How hard the
    the piano keyboard note was struct.

    NoteOff Ch Note Velocity

    Ch: 0-15. The channel that should
    terminate the note.

    Note: 0-127. The piano keyboard
    note to terminate.

    Velocity: 0-127. How quickly the
    piano keyboard note was lifted.

    Touch Ch Note Value

    Ch: 0-15. The channel sent
    the pressure information.

    Note: 0-127. The piano keyboard
    note that receives the pressure.

    Value: 0-127. The pressure value.

    CTouch Ch Value

    Ch: 0-15. The channel sent
    the pressure information.

    Value: 0-127. The pressure value.

    PWheel Ch Value

    Ch: 0-15. The channel sent
    the pitch wheel information.

    Value: 0-16383. Pitch wheel
    value. Center detent is 8192.

    PChange Ch Preset

    Ch: 0-15. The channel sent
    the preset change.

    Preset: 0-127. The new preset
    number to use.

    CChange Ch Ctrl Value

    Ch: 0-15. The channel sent
    the controller change.

    Ctrl: 0-127. The controller
    to change.

    Value: 0-127. The new value
    for the controller.
` ## MIDI File Commands `
    tempo value

    Translate trigger time values
    (in units of beats) to absolute
    time (in units of seconds).
`

## MIDI Files and SAOL In this section, we describe the MP4-SA semantics of MIDI File scores. MP4-SA supports MIDI File Format 0 and 1. #### Triggering Commands MP4-SA supports MIDI File scores with trigger times in units of beats. Absolute trigger times (units of seconds) are not supported. A MIDI File command executes when the trigger time is less than or equal to the current clock time of the SAOL simulation. The trigger time is converted from beats to seconds, using the current value of SAOL global tempo. As described in the last section, a MIDI File may include tempo commands. When a MIDI tempo commands executes, it changes the value of the SAOL global tempo. If a MIDI File does not include a tempo command at time zero, the SAOL global tempo takes on an initial value of 120 beats per minute (the MIDI File default). In Part III/3, we present a table that shows the place that MIDI commands trigger in the execution cycle. #### Channel Numbers As shown in the last section, most MIDI commands include a channel number. For Format 1 MIDI files, these commands may appear on different tracks, each of which has a separate "channel space". SAOL considers a MIDI command with channel number `N` that appears on track `M` of a MIDI Format 1 File to have the extended channel number `16*M + N`. Tracks in a MIDI File are numbered in order of appearance in the file, starting with 0. For a Format 0 MIDI file, the channel number of a MIDI command also serves as its extended channel number. |

## SAOL Instruments for MIDI A MIDI NoteOn command creates a new instance of SAOL instrument. SAOL instruments that are accessible by MIDI commands use a special declaration header, as shown on the right panel. MIDI-capable SAOL instruments have two instrument parameters. The first parameter is initialized with the note number specified by the instantiating NoteOn command. The second parameter is initialized with the velocity value of the NoteOn command. MIDI-capable SAOL instruments have the keyword preset following the parameter list, followed by a list of one or more MIDI preset numbers (separated by spaces). MIDI preset numbers are integers starting from zero. A particular present number may only be used by one instrument. A NoteOn command creates an instance of the instrument that has the same preset number as the extended channel of the command. Before the first NoteOn command executes on a MIDI channel, a PChange command should execute to set the preset number for the channel. If the MIDI channel requires a preset number greater than 127, the PChange command should be preceded by a CChange command for controller 32, to select a preset bank number. An instance created on a specific channel to play a specific note executes until a NoteOff executes with that that channel/note combination. At that time, the instrument is scheduled for termination at the end of the current execution cycle. Termination may be delayed in the following ways: Sustain Pedal. If the sustain pedal on the extended channel is depressed (i.e. MIDI controller 64 has a non-zero value) the termination for the instance is delayed until the sustain pedal is released (i.e. MIDI controller 64 is zero). Note that MIDI controller 64 is initialized to zero at the start of the simulation, and is changed by a CChange command. SAOL Intervention. As described in Part III/3, the SAOL extend statement may be used to delay the termination of an instrument instance. Instruments may use this technique to implement the release phase of the amplitude envelope of a note. | ## Example MIDI Headers `
    instr piano(note, vel) preset 13 2 {}
    instr sax(num, v) preset 300 {}
` ## Example MIDI Instrument `
    // tutorial sine wave
    // for MIDI

    instr mtone (num, vel) preset 0 {

      ivar a;     // freq const
      ivar scale; //  amp const
      asig x, y;  //  osc state
      asig init;  //  init flag

      // **********************
      // computed during i-pass
      // **********************

      // turns MIDI number into
      // oscillator constant

      a = 2*sin(3.141597*
    	    cpsmidi(num)/s_rate);

      // volume for tone

      scale = min(vel/127, 1);

      // **********************
      // computed during a-pass
      // **********************

      if (init == 0)
        {
          x = 0.25;
          init = 1;
        }

      x = x - a*y;
      y = y + a*x;

      output(y*scale);

    }
`

## Standard Names Five standard names contain information about MIDI: channel, preset, MIDIbend, MIDItouch, and MIDIctrl. The i-rate channel holds the extended channel number of the NoteOn command that created the instance, while the i-rate preset standard name holds the preset value of this extended channel. The k-rate MIDIbend standard name holds the value of the last PWheel command targeted to the extended channel of the instrument. If no PWheel commands have targeted this channel, MIDIbend has the value 8192 (associated with the center detent of the pitch wheel). The value of the k-rate MIDItouch standard name is updated whenever a CTouch command executes that targets the extended channel associated with the instance, or a Touch command executes that targets the note number and the extended channel number of the instance. The MIDIctrl array standard name holds the latest values of the 128 MIDI controllers. The array index chooses the controller number. A CChange command targeting the extended channel associated with the instance updates these controller values. Most MIDIctrl values are initialized to zero at the start of program execution; see the right panel for exceptions. Also, the MIDIctrl standard name may be used as an lval in a SAOL assignment statement. #### Non-MIDI Instances A SAOL program under MIDI control may have instrument instances that are not created by MIDI commands. These instances may be be created by SASL instr lines, or may be effects instruments created by global send statements. If a SASL or effects instance uses a MIDI standard name, the returned values reflect the status of the MIDI master channel. The right panel describes how MP4-SA determines the MIDI master channel. In addition, as we learn in the following chapter, instruments may be dynamically created in a SAOL program, using the instr statement. Accesses to MIDI standard names in a dynamic instrument reflect the MIDI channel of its parent instrument. | ## MIDI Standard Names `
    ivar channel;
    ivar preset;
    ksig MIDIbend;
    ksig MIDItouch;
    ksig MIDIctrl[128];

    The Slib library provides access
    to normalized and smoothed
    versions of the MIDI standard
    names.

    Slib also includes constants for
    scaling MIDI data, and preset maps
    for General MIDI.
` ## MIDIctrl Default Values `
    (only non-zero values shown)

    Number      Function      Value
    7           Volume         100
    10          Pan             64
    11          Expression     127
` ## The MIDI Master Channel `
    [1] If a real-time control
    source is present, extended
    channel 0 is the MIDI master
    channel. Else,

    [2] If streaming MIDI is
    encoded in the MP4-SA file,
    the first channel is the MIDI
    master channel. Else,

    [3] If a MIDI File is present
    in the MP4-SA file, the first
    track with MIDI commands is the
    MIDI Master Channel.
`

## Streaming and MIDI In the previous sections, we describe how a MIDI File may control a SAOL program. An MP4-SA binary encoding of this system would include the binary MIDI File data at the start of the MP4-SA file. The MP4-SA file format also supports distributing individual MIDI commands throughout the file, that do not have individual trigger times. These commands execute at the next opportunity for executing MIDI commands in the execution order, using the MIDI command semantics described in the previous sections of this chapter. Note that this distributed MIDI encoding is identical to to the data encoding on a MIDI port of an electronic keyboard. As a result, SAOL decoders may support real-time interactions with a MIDI keyboard using the framework described in this chapter. The `sfront` decoder includes an interface of this nature. Next: Part III/3: SAOL Instrument Control |

Copyright 1999 John Lazzaro and John Wawrzynek.