Abstract

The latent space of generative AI models affords unique creative possibilities and broad design space for AI-enhanced digital music instruments. Latent space navigation emerged as a new approach to sound synthesis that involves overriding latent vectors with sensor inputs capturing bodily motions and movements. Therefore, we see a timely opportunity to study how musicians perceive sound-producing gestures and tailor performance techniques in the audio latent space.

We present a user study workshop with an AI-enhanced digital music instrument with a tablet interface. Eighteen musicians were recruited to test out open-ended gestures and tasked to create musical scores. We contribute findings from an embodied music cognition perspective on how subjective perception of sound-producing gestures shapes musicians' technique development in audio latent space navigation. We discuss the implications of new gestural affordances discovered by participants in our workshop, aiming to elucidate new opportunities for digital musical instruments with audio latent space navigation.

[Upcoming] Zheng, S.J., Xambó Sedó, A. and Bryan-Kinns, N. (2025) ‘Exploring Gestural Affordances in Audio Latent Space Navigation’, Frontiers in Computer Science, 7. Available at: https://doi.org/10.3389/fcomp.2025.1575202