Overview
Four Theremin-type electric field sensors are incorporated by this Multimodal Music Stand (MMMS) along with a microphone and a video camera as input sources for the performance of computer music and electro-acoustic compositions.
Explanation
Using computer vision techniques like tracking the position of an instrument, the device was designed for interactive music performances, audio analysis and transformation of the performers playing. The device performs capacitive sensing through the four electric field sensors which tracks bodily gestures and movements to enhance expressivity. Without modifying the instrument or tether themselves with wires, the MMMS provides the performer added dimensions of musical control in order to influence the interactive aspects of a composition.
Accomplishing the interface to the computer is done by detecting timing signals using custom firmware for the CUI as well as sending the four proximity values to the host as continuous ranges of 16 bits each at an update rate of 100Hz. A full 3-dimensional input is being provided by the MMMS using 4 channels of capacitive sensing. The Z-axis gestural input is determined by the collective E-fields of all four antennas. The different sensing modalities are reinforced in this way to one another.
