The I-CubeX line of products aims to facilitate the design and implementation of environments and objects that respond to human actions such as walking, reaching, touching etc. as well as environmental parameters such as illumination, temperature, humidity etc.. The response can be in the form of sound, music, video, graphics, animation, robotic movement, etc.. Such responses are generated by 3rd party equipment. While it is a widely used tool for prototyping and experimentation wrt. human interfacing it is also perfectly suitable as a scientific research tool to gather data about movement and behaviour.

Application scope

The I-CubeX line of products only includes tools to capture the human actions and/or environmental variables and the tools to make these signals available to other equipment such as a computer or a musical instrument. To communicate with such equipment the MIDI data format is used, transmitted via MIDI cable, USB cable or Bluetooth wireless. MIDI is a widely implemented protocol for communications between musical equipment which has also found its way into video, graphics and animation software as well as various equipments, such as lights, used in theme parks and live shows. Even though I-CubeX uses the MIDI data format, tools are provided to output to devices that use other protocols, enlarging its application scope to gaming, engineering and science.

Components

To accommodate a wide variety of designs of such responsive environments and objects, the I-CubeX environment consists of the following modular components that can be configured in a multitude of ways:

One or more sensors.

One or more digitizers.

Configuration software.

Sensors

Sensors connect to a digitizer through a miniature, flexible cable with a 3-pin plug. Sensors can be positioned on the human body or on animals, on walls or floors or on sculptures. The multitude of sensors available from I-CubeX cover almost any human, animal or object motion, as well as a variety of environmental parameters. I-CubeX sensors output either analog voltages or digital I2C signals. I2C is a commonly used bi-directional protocol in robotics.

Digitizers

The digitizer converts the signal from the sensor to a digital message. A digitizer can convert up to 8 (USB-microDig, Wi-microDig, WiDig) or up to 32 (Digitizer) sensor signals. The Digitizer connects through a MIDI cable to a computer that is equipped with a MIDI interface (see figure 1) or directly to a MIDI capable device (see figure 2). The USB-microDig connects through a USB cable to a computer that is equipped with a USB port (see figure 3). The Wi-microDig connects wirelessly through Bluetooth Classic to a computer equipped with Bluetooth Classic (see figure 5). The WiDig connects wirelessly through BLE (Bluetooth Low Energy) to a mobile device or computer equipped with BLE (see figure 5 and 6) as well as through a USB cable to a computer that is equipped with a USB port (see figure 3). All digitizers use the MIDI data format to encode the sensor signals, but note that on a computer the USB-microDig, Wi-microDig and WiDig (when connected via USB) make this MIDI encoded sensor data available through a virtual serial port whereas the Digitizer makes the sensor data available through a MIDI port. Digitizers can operate in two modes: host mode, when the digitizer outputs raw, unprocessed sensor data as MIDI System Exclusive messages, and standalone mode, when the digitizer outputs processed sensor data as MIDi Channel Voice messages, using a variety of algorithms. These modes and other features are realized with the firmware, that functions as a kind of operating system and which runs on the digitizer.

Various software tools are available to work with I-CubeX (see figure 7). To configure the digitizer for standalone mode run the I-CubeX EditorX software. To configure the digitizer for host mode run the I-CubeX Link software. EditorX and Link communicate with the digitizer via a MIDI port. If the digitizer communicates via a USB or Bluetooth Classic serial port, the I-CubeX Connect software is used to represent the digitizer on the computer as a MIDI port. Connect also functions as a bridge using OSC (Open Sound Protocol, a common IP-based communication protocol) to other applications, possibly running on other computers, or Webhooks for IoT (Internet of Things) applications via the internet. When communicating via BLE (Bluetooth Low Energy) with a digitizer, the BLE-MIDI protocol is used (see figure 8). EditorX sets a variety of parameters in the digitizer such as the type of processing of the sensor signal and which MIDI message to map to. It provides a way to verify if the sensor captures what is required to generate an appropriate response. Once the user has configured the digitizer with EditorX as needed, the MIDI messages can be sent directly to MIDI-capable software, such as GarageBand, Ableton Live or TouchDesigner, or a hardware device to generate a response. In the standalone mode of operation it is then not necessary to use a computer program to process or analyze the sensor data. Using EditorX or Link the sensor data can also be saved on disk for later analysis using 3rd party applications. Our Keys software enables the user to convert the sensor data intelligently to keystrokes, which then in turn can control other software programs such as media players. Link provides a convenient way to configure the digitizer for host mode, and process, analyze, display, record and map the raw sensor data to create more complex output. Plugins for 3rd party software and tools to integrate I-CubeX with a number of programming environments, such as Max, or languages such as C/C++, are available as well. Figure 8 illustrates communication via WiFi with the WiDig using OSC, HTTP and web socket protocols. Configuration of the WiDig takes place using a web browser.

I-CubeX arose out of a research project in 1995 directed by Axel Mulder, while he was doing his PhD on virtual musical instruments at the Department of Kinesiology, Simon Fraser University, to address the need for better tools for artists to create interactive art and for musicians to more easily create or modify musical instruments. The project led to a paper entitled The I-Cube System: moving towards sensor technology for artists, after which it was commercialized.