Mass-Spring-System model for real time expressive
behaviour synthesis
Why and how to use Physical Model in Pure Data (pmpd)
Cyrille Henry (ch@chnry.net)
Abstract
Mass-spring-system (MSS) physical model (PM) are vastly documented in literature and
on internet. They where developed to imitate natural behaviours, specially for audio
synthesis but they can also be used to create natural behavious or even “plausible”
behaviours. This “plausible” behaviours are comportment that does not imitate any
specific real life behaviours, but - since basic real life physic are respected - looks
“natural”. This is specially useful for real time user interactions. Even if this “plausible”
behaviour can be use for any human / computer interface this document will mostly
focuses on artistic application.
After a quick overview of the equation used for this kind of simulation, this document will
list different strategies use in MSS (implementation, creation of a simulated system,
strategies for interaction and using data from this system). Finally the pmpd library for pure
data will be presented.
Introduction
In this document, we discuss the use of mass-spring-system physical model to create
expressive movement for real time interaction.
We are dealing with the production and the control of data flow aiming at the control of any
kind of software parameter : Sound synthesis, images synthesis, lighting or motor actuator
by example can be driven thanks to a physical model. This parameters can usually benefit
from an expressive control. We are using physical model to synthesize data flow and
control their behaviours.
MSS model can be use to imitate natural behaviours, but they can also be used to create
lot's of different behaviours. We did not focuses on real behaviours, but on “plausible”
behaviours : a movement that look natural, even if it did not imitate a specific real life
physics. MSS model can easily generate this “plausible” behaviours since it respect basic
real life physics. Moreover this kind of physical models allow a wild range of user
interaction for real time control of the movement.
Using this models can lead to control of different media with data flow and to control
correlation between this data, thus having sound and images by example that does not
represent the same data, but express the same underling process.
This document aim is to give key to a deep understanding of process involved in MSS
model. After a quick overview of the equation used for this kind of simulation, we will
present different approach to create topologies of mass / link network. Then it will list
different strategies used to interact with this MSS simulation. Latter, we will list few
approach to use the generated data flow for both sound synthesis control and images
synthesis control. It should not be seen as exhaustive, but as a summarized of the author
experience in developing PM for many different artistic applications. Latter, we will have a
look at pmpd : a specific implementation of MSS system for the Pure Data software.
Index
Abstract.................................................................................................................................................1
Introduction..........................................................................................................................................1
Index.....................................................................................................................................................2
Motivations...........................................................................................................................................4
Real motion......................................................................................................................................4
Movement using a simple interpolation...........................................................................................5
Mass Spring System.........................................................................................................................5
Physical Model.....................................................................................................................................7
Basic equation..................................................................................................................................7
Implementation................................................................................................................................8
Units.................................................................................................................................................8
Basic oscillating system...................................................................................................................8
Topologies........................................................................................................................................9
Regular structures.......................................................................................................................9
Random structures.....................................................................................................................10
Collection of independent simple structures.............................................................................11
Shape morphing.........................................................................................................................11
Modal synthesis.........................................................................................................................12
Real time interaction strategies......................................................................................................12
Moving fixed masses................................................................................................................12
Sending force to masses............................................................................................................12
Changing physical parameter....................................................................................................13
Using data created by a MSS.........................................................................................................13
Sound synthesis.........................................................................................................................14
Direct audio simulation........................................................................................................14
Scanned synthesis.................................................................................................................14
“Many element synthesis”....................................................................................................15
Visual synthesis.........................................................................................................................15
Complex system.............................................................................................................................17
Non linearity.............................................................................................................................17
Space limitation of links.......................................................................................................17
Non linear rigidity................................................................................................................17
Non linear damping..............................................................................................................18
More interactions......................................................................................................................18
Motion damping...................................................................................................................18
Creep of a link......................................................................................................................18
Plastic link............................................................................................................................19
Breaking link........................................................................................................................19
Link elongation computation error...........................................................................................19
MSS simulation and emergent phenomena...............................................................................20
Instabilities................................................................................................................................21
Controlling instabilities.............................................................................................................21
Example : chdh – emergence....................................................................................................23
MSS and Pure Data: the pmpd library................................................................................................26
Installation.....................................................................................................................................26
Generality.......................................................................................................................................26
Masses............................................................................................................................................26
Link................................................................................................................................................26
Metronome.....................................................................................................................................27
Interactor........................................................................................................................................27
Test.................................................................................................................................................27
pmpd* objects................................................................................................................................27
System creation.........................................................................................................................28
Interacting with the system.......................................................................................................28
Getting data from a system.......................................................................................................28
Pmpd*~ objects..............................................................................................................................29
website...........................................................................................................................................29
Sources...........................................................................................................................................29
Installation.....................................................................................................................................29
Conclusion..........................................................................................................................................29
Annexes..............................................................................................................................................30
References......................................................................................................................................30
pmpd objects list............................................................................................................................31
Motivations
Real motion
Since a movement is initiated thanks to a force, looking at the speed and acceleration (that
are linked to force, as we will see latter), is a good way to analyses a movement, and gives
more details than only looking only at the position over time. Figure 1 represent position,
velocity and acceleration over time for 6 different recording of a hand motion moving a
cursor from 2 different points 10 cm approximatively apart.
Even for simple movement with relatively similar position vs. time curve, lot's of variety can
appears in real life motion. When looking at the acceleration curve : minimum and
maximum acceleration and position of this extreme can be relatively different. The human
brain have been trained to analyse and understand this differences. Being able to
reproduce such diversity is the first step toward generating expressive behaviours.
Figure 1: 6 recording of a simple hand movement, and there corresponding velocity and
acceleration over time.
Movement using a simple interpolation
Using Pure data to move a parameter is usually done with the [line] object. This object
perform a linear interpolation between two points (see figure 2). This is not a kind of
movement that can easily be find in the nature. Using this to move a fader result in a very
different movement than what would be performed by a human, mainly because it did not
respect the world physics. The linear interpolation imply a constant speed of the
parameter, and no velocity before and after the transition. This result in an infinite
acceleration at the beginning and at the end of the movement.
Such movement is easily identifiable as artificial.
Mass Spring System
Fdili Alaoui and Al explain motivation using a MSS in an article published in the IJPADM
revue (REF 1). Here is an extract :
MSS present various properties that make them attractive to use in the performing arts
(Hsieh 2007; Jacquemin 2008; Johnston 2009; Fdili Alaoui 2012). They have the ability to
produce physically plausible behaviours because they are governed by the laws of physics
and hence have an evocative representation of real-world dynamics. We describe the
behaviour of a physical model as physically plausible when it generates realistically
perceived dynamics that are created autonomously to evoke real-world dynamics, without
imitating real-world mechanics.
MSS provide a wide range of controllable parameters in real-time (such as the parameters
of the forces applied to the masses) that govern its behaviours, and thus offer rich
possibilities of interaction. The MSS can be controlled through the mapping between the
MSS physical parameters and the features of the analysed movement. The resulting
interaction would create a relationship between the performer’s movement and the
associated MSS behaviours. This relationship follows pre-defined rules or interaction
scenarios.
figure 2: movement generated by a linear interpolation
MSS simulations generate data streams that can control different media. The applications
described in this article are limited to sound and visual media, but other actuators such as
lighting devices can also be controlled. The data generated can be directly used as a
synthesis kernel. For example the ‘scanned synthesis’ method by Verplanck et al. (2000)
uses MSS to generate shapes, which build waveforms for audio synthesis. The use of
MSS for a parallel control of different media can ensure a strong recognisable coherence
between these media. The same physical simulation can generate data concurrently
synthesized into sound and graphics. In real life a moving object’s velocity and
acceleration produce sound while the viewer perceives the object’s position in time. Thus
to generate plausible cross-media behaviours, one can simulate real life cross-modal
phenomenon and use the mass positions to control the graphical rendering and their
velocities and accelerations to control the sound synthesis. The parallel multimedia control
allows us to perceive the audio-visual phenomenon with more consistency as both sound
and visual behaviours come from the same physical simulation.
Figure 3 represent few movement that was created using simple MSS. This movements
present few particularities that can be found in real life motion presented earlier.
figure 3: different movement generated by a MSS
Physical Model
Basic equation
MSS simulation are based on simple idealized physical elements :
- The mass is a punctual mass : it got weight, so it is subject to inertia, but it does not have
size, so it is not subject to rotation. The mass react to newton equation :
∑F = m γ
The mass is the moving part of the simulation. It react to input forces and output it's
position. It's weight is the only adjustable parameter.
- A spring create an elastic connection between two masses. The spring create a force
inverse to it's elongation or compression.
F = K * ((x2 – x1) - l0)
The amplitude of the created force is proportional to the spring rigidity (K). It can also have
a specific size (l0). The spring have no mass.
The spring react to two masses positions, and interact with them sending two opposite
forces. It allow kinetic/potential energies conversion, thus kinetic energy transfer from a
mass to another.
- Just like a spring, a damper connect two masses. The masses positions are used to
compute forces thanks to the viscosity (D) parameter of the damper. The force is
proportional to the elongation speed of the damper.
F = D * (v2 - v1)
The damper is the only way for the system to lose energy.
Both spring and damper can be aggregate in a single visco-elastic link.
This equation can be computed in a one dimensional space : i.e. masses can only be
moved in 1 direction and link are also projected in this direction. But they can also be
computed in a 2, or 3 dimensional space, and they can also be easily extrapolated to a
higher dimensional space.
figure 4: representation of a spring and a damper connected as a single link
Implementation
In order to create a MSS simulation, one have to implement this equations on a computer.
In the digital world, states of the model are computed in discreet time. i.e. displacement of
a mass is not a continuous function, but a finite serial of point: xn.
Position at time t is noted : xt, while xt-1 is the position at previous time step.
The aim is to compute position of all masses at time t+1 depending on variables available
at time t.
we usually compute velocity (v):
vt = xt-xt-1 / dt
and acceleration (γ):
γt = vt-vt-1 / dt
In order to simplify the simulation, all constant can be removed : dt (time difference
between each computational step) is usually chose to be 1.
Since (for a mass) :
∑F = m γ
so
vt+1 = vt + ∑F/M
and
xt+1 = xt + vt+1
For the visco-elastic link (x1 and x2 are the position of the 2 masses connected by this
link) :
F1t = K(x2t-x1t) + D(v2t-v1t)
and
F2t = -F1t
One can noticed that changing weight of the masses in the same proportion as rigidity and
damping of links did not change the simulation.
Some implementations uses only xt and xt-1, while other use xt and vt. The equation based
on xt and xt-1 are not provided but can be very easily deduced from the latter.
A PM simulation is based on a set of masses ans links. Links use masses position to
compute the force they generate. Then, every masses add the forces they receive to
compute a new velocity and a new position. This new position is send to the links in order
to start again computing the forces...
Units
pmpd does not use specific units, you can chose your own. However, you should try to use
units you that are consistent. i.e: you chose inches for the distance unit, then rigidity unit
should be force unit divide by inches. For application describe in thius document, we just
avoid using units.
Things are relative: a mass weighting 10 will react the same way to a force of 1 than a
mass of 100 to a force of 10.
Basic oscillating system
Simple model do need only a very small processing power and implementation can be
very simple. But they could be in great use in lot's of different applications.
One of the simplest examples of MSS consists of a single mass connected to a fixed point
with a single link, forced to move along a single axis, and governed by an elastic and a
viscous force. When the mass is displaced from its equilibrium position, its dynamics is
controlled by the sum of an elastic force proportional to the displacement and a viscous
force with a damping proportional to its velocity. When the mass is released, it oscillates
with a fixed resonant frequency around the equilibrium position and with decreasing
amplitude over time. Such a system is called a simple damped oscillator and the position
of its mass over time is illustrated in figure 5.
Despite its simplicity, this system can be used in various ways. For example, when moving
the fixed point, the mass position acts as a low pass resonant filter, while the spring
elongation is a high pass filter.
When two simple damped oscillators are combined, the system’s oscillations are more
complex and result from the interaction of the two frequencies of each single oscillator.
Thus, a MSS combining dozen, hundreds, or thousands of damped oscillators or more
interconnected masses, can generate very complex behaviours that can then be used to
metaphorically evoke human movements or to control any digital process. From a
frequency point of view, the more masses composes the system, the more vibrating mode
the system can admit : i.e. a 1 mass system can oscillate freely on 1 single resonant
frequency, a 2 masses system can admit 2 frequency etc. Complexity usually arise with
the masses number (or non linearity as we will discuss later).
Topologies
In order to use a MSS physical model, one have first to create a topology. i.e. a group of
mass and connection between this masses. The topology can try to imitate a natural
structure in order to mimic it's behaviours, but since we are also interested in abstract
behaviours so we will focus on all kind of structure.
Regular structures
One of the easiest structure to experiment with are regular structure : string, nets or stack.
The way masses are connect to each other is important for the global behaviours. For a 1d
model by example, like the one presented on figure 6, the red connection are link between
a mass and it's 2 neighbour and defined a string like system.
figure 5: The mass position of a simple damped oscillator after having been displaced from its
equilibrium position.