Technical Reports - Query Results

Effects of musical expertise on audiovisual integration: Instrument-specific or generalisable?

Laura Bishop, Werner Goebl

During ensemble performance, musicians exchange auditory and visual signals that can help in
synchronising with each other's actions. Musicians must integrate corresponding auditory and visual
signals accurately to make use of them. Precision in audiovisual integration improves with
increasing perceptual-motor expertise, perhaps because experts are better able to predict when the
auditory effects of observed actions should occur. Performance expertise has been found to have a
greater effect than visual expertise on musicians' prediction abilities during synchronisation tasks,
with performers better able to synchronise with actions that fall within their own motor repertoires
than with actions they have only ever observed. It is unclear whether the effects of expertise on
audiovisual integration are likewise instrument-specific or generalisable across instruments,
however. The present study investigated the potential instrument-specific effects of expertise on
audiovisual integration. Expertise in playing a particular instrument was hypothesised to facilitate
prediction of observed actions, increasing sensitivity to audiovisual asynchrony in that instrumental
context. Ten-second clips were extracted from audio-video recordings of clarinet, piano, and violin
performances, and presented to highly-skilled clarinettists, pianists, and violinists. Clips either
maintained the audiovisual synchrony present in the original recording or were modified so that the
video led or lagged behind the audio. Participants indicated as quickly as possible whether the audio
and video channels in each clip were synchronised. Sensitivity to audiovisual asynchrony was
assessed for each expertise group/stimulus instrument pairing by evaluating the mean point of
subjective synchrony and the mean range of asynchronies most often rated as synchronised (i.e.
temporal integration window; TIW). Though participants across expertise groups detected
asynchronies most readily in piano and least readily in violin stimuli, pianists performed
significantly better for piano than for clarinet or violin. A relationship between musical training and
TIW was also observed with data pooled across stimuli. Thus, sensitivity to audiovisual asynchrony
improved generally with increasing expertise, and only pianists showed facilitation for their own
instrument. Sensitivity to audiovisual asynchrony was affected by musical training and the nature of
sound-producing movements observed. The results suggest that, to some extent, the effects of
performance expertise can be instrument-specific, though they may generalise across instrumental
contexts more readily during audiovisual asynchrony detection tasks than during synchronisation
tasks, when overt, precisely-timed movements are required.

Citation: Bishop L., Goebl W.: Effects of musical expertise on audiovisual integration:
Instrument-specific or generalisable?, in Proceedings of the 13th International Conference on Music Perception and Cognition, Seoul, South Korea, 2014.