Developments in Music Technology have seen major changes in the manner in which artists, performers, and creatives interact with digital technology; this is arguably due to the increasing variety of digital technologies that are readily available today. Digital Musical Instruments (DMIs) present musicians with performance challenges that are unique to Computer Music. One of the most significant deviations from conventional acoustic musical instruments is the level of physical feedback conveyed by the instrument back to the user. Currently, new interfaces for musical expression are not designed to be as physically communicative as acoustic instruments. DMIs are often void of physical feedback and therefore lack the ability to impart important performance information to the user. Moreover, there is currently no standardised way to measure the effects of this deficit. In a design context, best practice would expect that there should be a set of methods to effectively, repeatedly, and quantifiably evaluate the various elements of functionality, usability, and user experience involved in a DMI interaction. Earlier applications of haptics have tried to address device performance issues associated with the lack of feedback in digital device designs and it is argued that the level of haptic feedback presented to a user can significantly affect the user’s overall emotive feeling towards a musical device. In my research I explored a number of techniques in which physicality could be reintroduced to digital interactions with musical devices. I conducted psychophysiological studies that measured the effects of vibration, designed an evaluation framework that could be applied to musical instruments, and presented functional and longitudinal studies that applied the framework in the evaluation of haptics applied in Computer Music.

This chapter presents an investigation that compares the performance of game controllers in two-dimensional pointing tasks as defined in the international standard that specifies the requirements for nonkeyboard input devices, ISO 9241-9. In addition, we discuss the evaluation of usability and user experience with these devices during gameplay. We compared performance measurements for controllers while varying the user’s exposure to the different feedback elements contained within each controller device. We assessed the performance of the controllers according to the ISO 9241-9 evaluation recommendations. The devices used in the study included a Logitech mouse and keyboard, a Logitech Bluetooth Touchpad and keyboard, a Sony Playstation DualShock 4 controller, and Valve’s first-generation Steam controller. Besides performance testing, we measured user experiences with the controllers while playing a popular first-person video game. Participants were asked to complete game levels for each type of controller and answer questions outlining their experience.

We present the findings of a pilot-study that analysed the role of haptic feedback in a musical context. To closely examine the role of haptics in Digital Musical Instrument (DMI) design an experiment was formulated to measure the users’ perception of device usability across four separate feedback stages: fully haptic (force and tactile combined), constant force only, vibrotactile only, and no feedback. The study was piloted over extended periods with the intention of exploring the application and integration of DMIs in real-world musical contexts. Applying a music orientated analysis of this type enabled the investigative process to not only take place over a comprehensive period, but allowed for the exploration of DMI integration in everyday compositional and explorative practices. As with any investigation that involves creativity, it was important that the participants did not feel rushed or restricted. That is, they were given sufficient time to explore and assess the different feedback types without constraint. This provided an accurate and representational set of qualitative data for validating the participants’ experience with the different feedback types they were presented with.

We present an experiment that investigated the effect of vibrotactile stimulation in auditory pitch discrimination tasks. Extra-auditory information was expected to have some influence upon the frequency discrimination of auditory Just Noticeable Difference (JND) detection levels at 160 Hz. To measure this, the potential to correctly identified positive and negative frequency changes for two randomly divided groups was measured and then compared. The first group was given an audio only JND test and the second group was given the same test, but with additional vibrotactile stimulus delivered via a vibrating glove device. The results of the experiment suggest that in musical interactions involving the selection of specific pitches, or the detection of pitch variation, vibrotactile feedback may have some advantageous effect upon a musician’s ability to perceive changes when presented in synchrony with auditory stimulus.

Here we present guidelines that highlight the impact of haptic feedback upon the
experiences of computer musicians using Digital Musical Instruments (DMIs). In this context, haptic feedback offers a tangible, bi-directional exchange between a musician and a DMI. We propose that by adhering to and exploring these guidelines the application of haptic feedback can enhance and augment the physical and affective experiences of a musician in interactions with these devices. It has been previously indicated that in the design of haptic DMIs, the experiences and expectations of a musician must be considered for the creation of tangible DMIs and that haptic feedback can be used to address the physical-digital divide that currently exists between users of such instruments.

Here we present experimental results that investigate the application of vibrotactile stimulus of pure and complex waveforms. Our experiment measured a subject’s ability to discriminate between pure and complex waveforms based upon vibrotactile stimulus alone. Subjective same/different awareness was captured for paired combinations of sine, saw, and square waveforms at a fixed fundamental frequency of 160 Hz (f0). Each arrangement was presented non-sequentially via a gloved vibrotactile device. Audio and bone conduction stimulus were removed via headphone and tactile noise masking respectively. The results from our experiments indicate that humans possess the ability to distinguish between different waveforms via vibrotactile stimulation when presented asynchronously at f0 and that this form of interaction may be developed further to advance digital musical instrument (DMI) extra-auditory interactions in computer music.

We present an analysis of literature relating to the evaluation methodologies of Digital Musical Instruments (DMIs) derived from the field of Human Computer Interaction (HCI). We then apply choice aspects from these existing evaluation models and apply them to an optimised evaluation for assessing new DMIs.

This experiment is a case study that applies a HCI-informed DMI Evaluation Framework. This framework applies existing HCI evaluation methods to the assessment of prototype Digital Musical Instruments (DMIs). The overall study will involve a three-part analysis: a description and categorisation of the device, a functionality evaluation that included an examination of usability and user experience, and finally an exploration of the device’s effectiveness as a digital instrument. Here we present the findings of the first two parts of the framework, outlining the constituent components of the interface and testing the functionality of the device. The final stage of analysis will involve a longitudinal study, and will be carried out in order to assess the musical affordances of the device.

This paper introduces the Audio-Tactile Glove, an experimental tool for the analysis of vibrotactile feedback in instrument design. Vibrotactile feedback provides essential information in the operation of acoustic instruments. The Audio-Tactile Glove is designed as a research tool for the investigation of the various techniques used to apply this theory to digital interfaces. The user receives vibrations via actuators distributed throughout the glove, located so as not to interrupt the physical contact required between user and interface. Using this actuator array, researchers will be able to independently apply vibrotactile information to six stimulation points across each hand exploiting the broad frequency range of the device, with specific sensitivity within the haptic frequency range of the hand. It is proposed that researchers considering the inclusion of vibrotactile feedback in existing devices can utilise this device without altering their initial designs.

Digital Publications:

An electronic edition comprised of a selection of scholarly reflections from doctoral candidates on the Digital Arts & Humanities Ph.D. programme, part of the Digital Academy at University College Cork, Ireland.