Music and cross-modal associations
Table of contents
Project description
In this project, we explore the intriguing world of cross-modal associations involving auditory stimuli by examining how people from different cultures perceive connections between music and images, shapes, colours, emotions, and words. Participants from across the world will help understand how musical features, such as timbre and melodic contour, are associated with non-auditory features.
This research could provide insights into how universal elements of music perception, cultural, and musical background shape our experience of music. By delving into the intricate relationships between sound, images, and language, the project seeks to enhance our understanding of human perception and cultural diversity in the realm of music.
The first publication related to this project can be found here (open access):
Di Stefano, N., Ansani, A., Schiavio, A., & Spence, C. (2024). Prokofiev was (almost) right: A cross-cultural investigation of auditory-conceptual associations in Peter and the Wolf. Psychonomic Bulletin & Review, 31, 1735–1744. https://doi.org/10.3758/s13423-023-02435-7
The second publication can be found here (open access):
Di Stefano, N., Ansani, A., Schiavio, A., Saarikallio, S., & Spence, C. (2025). Audiovisual Associations in Saint-Saëns’ Carnival of the Animals: A Cross-Cultural Investigation on the Role of Timbre. Empirical Studies of the Arts, 43(2), 1162-1180. https://doi.org/10.1177/02762374241308810
We have also published (open access) in collaboration with Dr. R. D. Wanke (Centre for Interdisciplinary Studies; University of Coimbra, Portugal):
Wanke, R., Ansani, A., Di Stefano, N., & Spence, C. (2025). Exploring auditory morphodynamics: Audiovisual associations in sound-based music. i-Perception, 16(4). https://doi.org/10.1177/20416695251338718
Furthermore, two articles are currently under review. In one, we explored the cross-modal associations of the same compositions in 6 to 9-year-old children. In the other one, we attempted to cross-culturally map the 12 musical intervals onto several extra-auditory dimensions (e.g., gustatory, spatial & physical attributes, emotional/affective, tactile, colour, and shape).
The work carried out in Music and Cross-Modal Associations laid the foundation for a new project funded by the Research Council of Finland. Building on the same research themes and experimental approaches, the new 4-year project - FRAMES (Fine-gRaining Audiovisual cross-Modal corrESpondences [9/2025-8/2029]; PI: Dr Alessandro Ansani) - has been awarded an Academy Research Fellowship grant. FRAMES will further investigate how people make sense of complex auditory and visual input through online psychophysical studies and lab-based eye-tracking experiments. Furthermore, FRAMES will generate substantial datasets, which will be openly shared to support transparency and reproducibility. The expected results include a more systematic categorisation of crossmodal correspondences, insights into implicit and explicit attentional mechanisms, and the development of tools to facilitate research in this field.
FRAMES benefits from two invaluable collaborations: one with the Italian National Research Council’s Institute of Cognitive Sciences and Technologies (ISTC-CNR), through Dr Nicola Di Stefano, and another with the University of Oxford, through Prof Charles Spence.
As soon as FRAMES officially starts, this page will be updated with links to detailed information and materials related to FRAMES.
