The application has two modes; the wand's right button alternates between them. Exit the application by pressing the escape key (as with most CAVE apps).
In the first mode, the wand moves the particle (ball) around. A corresponding "light saber" sound moves with it and is modulated by the ball's acceleration, sonifying the "force" (F = ma) applied to it. To draw an orbit, (click and) release the left button to release the ball. If you move the wand while clicking the left button, you will throw the ball in that direction. Use the joystick to move the particle closer or farther away, to effectively change the length of your throwing arm and the speed with which you can throw the ball.
In this first mode, then, the set of orbits you can draw has six degrees of freedom: three from the particle's initial position and three from its initial velocity.
In the second mode, the orbit is recomputed continuously based on the wand's orientation. To change the orbit, point and roll the wand in different directions. The set of orbits now available has, strictly speaking, only three degrees of freedom (azimuth, elevation, roll) drawn as red bars on the pedestal. But this set has greater variety than the earlier 6-DOF family because it is taken from the full 14-DOF family through a high-dimensional interpolation technique. The 14 parameters defining an orbit's shape are drawn as blue bars on the pedestal, and you can see all 14 changing as you change the 3 red bars.
This 3-D to 14-D mapping is defined by a simplicial interpolator. I just chose eight shapes which I thought were interesting and varied; the interpolator did the rest, creating a continuous 3-D control space which interpolates between these shapes. So the 3-DOF set is a warped, crinkled 3-D slice of 14-dimensional space which passes through interesting points; this is why it exhibits more variety than the linear 6-D subspace available in the first mode.
If you find crinkled 3-D slices of 14-D space weird, think of the previous
paragraph like this:
I chose eight sets of values for the 14 blue sliders which produced
pleasing shapes. Each set (each shape) was attached to a particular
orientation of the wand; as the wand points in directions intermediate
between these eight particular directions, the blue slider values
smoothly change between the eight initial sets of values I chose
(the shape smoothly changes between the eight initial shapes).
(Technically: the relative distance matrix of the 8 points in R^14 is duplicated as closely as possible in R^3 by a genetic algorithm to define eight image points. The Delaunay triangulation (tetrahedralization) of these points is induced back into R^14 as a simplicial 3-complex. The mapping from R^3 to R^14 is defined pointwise by finding in which simplex a given point lies, and then computing the point's barycentric coordinates in that simplex. Having this data, we choose the point in the corresponding simplex in R^14 with corresponding coordinates. This method works for arbitrary dimensions and numbers of points, and the interpolation runs in constant time independent of the number of dimensions, number of points, or size of the simplicial complex.)
The sound in this second mode is driven only indirectly: it changes quickly or slowly as the shape of the orbit changes quickly or slowly. An ad-hoc definition of shape is used to compute the perceptual difference between two successive orbits as a pair of scalars, drawn as yellow and blue bars on the right side of the CAVE. When the orbit is changing slowly or not at all, the music slows down in literal rhythm and harmonic rhythm. When the orbit changes quickly (e.g., by flailing the wand around), the notes speed up and the harmony becomes unstable. (Technically: rhythm is a Poisson process of varying rate, and harmony is a set of pentatonic chords going around the circle of fifths, with an amplitude envelope on tessitura like Shepard tones.)
The laws of motion are a generalization of the so-called
harmoniograph,
a mechanical device which draws
decaying spiral-like objects like the one below with a pen on paper.
The title of this composition comes from a pedagogical observation by the person who introduced the harmoniograph to me, my high school geography teacher Tom McEwen: If it moves, it will capture children's attention. If it lights up, the same. If it does both, you've got them completely.
If you have a VRML 2.0 viewer, click on the image at right of a harmoniograph
model to examine it from different angles.
The immediate application of this research is the control of algorithms which compute acoustic waveforms in real time (such as the sounds in this piece). For many years, computer music was done in "batch mode", but current computers are fast enough to compute interesting sounds in real time. This software lets us control these sounds' parameters simultaneously from the limited bandwidth of human gestural input. But as shown here, the interpolation technique is applicable to real-time systems beyond acoustic ones: VR scene graph and lighting design, battlefield strategy, dynamic load balancing, high-dimensional visualization, etc.
If you don't have a subwoofer, turn up the bass: much of the sound is at very low frequencies.