A Violin Controller for Real-Time Audio Synthesis

Camille Goudeseune
1999-04-20, revised 2000-08-23, links revised 2004-09-14 and 2014-08-21.

(2004 Aug. 28)

(This document is at http://zx81.isl.uiuc.edu/camilleg/eviolin.html.)

Movies (Quicktime 4 or later):
Study #1, 45 MB
Study #2, 39 MB

1. System Overview

This "violin controller" is used not as a sound-producing device, but as a gesture-input device. It is no more a violin than a Yamaha synthesizer is a piano: both take standard gestures as input, both produce nonstandard sounds as output. (With this disclaimer out of the way, I'll still continue to call it a violin in what follows.)

The violin's bridge produces a signal with varying pitch and loudness (depending on all the usual violinist's bowing and fingering tricks); a computer measures this changing pitch and loudness and uses these changing numbers to affect an (entirely different) sound being simultaneously computed and played.

Another gesture input is spatial: the position and orientation of violin and bow are measured on the fly, producing another stream of changing numbers which can modify the sound.

Particular systems used:

1.1 The Jensen violin

Quick facts about my model (quite a variety is available from just this manufacturer, never mind others like Zeta and Jordan):

General playing observations

1.2 Virtual Sound Server (VSS)

VSS is both the sound synthesis engine and the glue that holds the whole system together.

For this project I run VSS on a generic Intel Linux box: an SGI is still too expensive to dedicate to this single project, and MS Windows has unacceptable latency (130 msec) for real-time audio. Linux (Red Hat 5.2, with ALSA sound drivers) is fast, cheap, and reliable.

What VSS is and does... that's a long story. Try the tutorial or the reference manual.

1.3 The "fiddle" pitch tracker

Miller Puckette's published code does polyphonic tracking of pitch and amplitude. I've seen the algorithm guess pitches incorrectly for enough cases of double- and triple-stopping that for now I've left it as monophonic. In this project it runs as part of VSS -- I extricated it from Puckette's original Pure Data implementation.

I also made a few changes to its behavior:

Tracking latency is around 1/30 second.

1.4 The SpacePad motion tracker

Full motion capture requires fancy hardware like this. This system matches the demands placed on it by the violin: low weight, low restriction of mobility, low tracking latency, high accuracy, small size, and low cost ($1400). For this you get an ISA card, two "receivers" whose position is measured, manuals and software.

Fancier systems from Ascension, Polhemus and InterSense cost three to thirty times as much and require much coddling. (Prices in this industry are not falling much; the SpacePad is 4-year-old technology, in fact.)

Simpler systems have been designed around mercury switches and force-sensing resistors (particularly for dancer interfaces) and radio field sensing (Mathews Radio Drum, MIT projects like the Sensor Chair). Most recently, micromachined accelerometers show promise as very inexpensive motion sensors. Dan Trueman has experimented with them in his R-Bow. Their main disadvantage is that they cannot distinguish between acceleration and gravity. Dan says, "If you are bent way over and bowing real fast, who knows what those things are saying...". If you try to work around it and do the math, you'll find that you need three separate triaxial accelerometers to track position accurately, which is actually more expensive than the SpacePad and much more work.

The SpacePad tracks the position and orientation of two sensors (where they are and where they're pointing). It does so by generating a time-varying magnetic field from an antenna you're instructed to make (my prototype portable model is made from 30 feet of speaker wire, a garbage bag, some dowels and string, and looks like a stunt kite). The field sensors are little half-inch boxes on the end of 10-foot cables. One is taped to the bottom of the violin; the other is sewn into a fingerless glove worn on the violinist's bowing hand.

Ideally, the positions and orientations are measured with 13-bit resolution. This falls somewhat because the size of the measured performing space is not adjustable (hey, it's a "budget" system), and because large metallic objects nearby (furniture, rebar in concrete floors, music stands) distort the magnetic field. Accuracy remains well above the 7 bits provided by MIDI, though.

Tracking latency is 1/60 second, half that of video-based systems. If we track only one sensor, the latency drops to 1/120 second.

The drivers for the ISA card are written for MS-DOS 3 using an old Borland compiler. Ascension graciously released the source code to me, and I ported the drivers to Linux using gcc.

Visual feedback is given to the performer with a single-pixel color display, mounted near the bridge so it is always at the edge of the performer's field of view. The brightness and full-spectrum hue can be updated with a latency of 3 to 7 msec, to warn the performer that they are approaching a boundary in the space (or, in a simpler demonstration, to tell them if they're playing sharp or flat!). Its location on the instrument itself, its simplicity (no dials or numbers to read -- even focusing isn't necessary), and its fast response make it intuitively feel like part of the instrument. The display is implemented with an array of red, green, and blue LEDs mounted in a light diffuser and driven by an 8-bit latch from the Linux PC's parallel port.

1.5 The CAVE motion tracker / audio-visual display

The SpacePad is portable to the concert stage, but I'm also investigating what a full virtual environment offers. I've implemented an environment where the violinist can move from place to place to play "into" several microphones. The microphones are simulated, of course; each mic's pickup pattern is actually a long cylinder, 2 feet wide, which very slowly moves around the CAVE. Each mic feeds a filter/echo combination.

As such a dynamic environment is overwhelming at first, it has a "complexity slider" which gradually introduces features as it is turned up. When the violinist plays at the tip of the bow, the echoing sounds suddenly drop out. The complexity slider also controls which sound goes into the microphones, the natural sound of the violin or a synthesizer controlled by the violin.

2. Demonstrations

Three technical terms: These examples proceed from very simple extensions of standard violin technique through more advanced demonstrations of what is possible when you put software between the performer's physical input and the instrument's acoustic output.