ENSEMBLE
Enhanced Sonic Perception and Communication through Sensory Substitution
Independent Project
Timeline: September 2022 - May 2023
Tools & Skills:
Problem Definition, User Research & Testing, Prototyping, 3D Design & Printing, Laser Cutting, Casting, Mechanical & Electronics Design.
ESP32, IoT, Python, C++, EEG, TouchDesigner, Fusion 360.
Amidst a world of rich diversity, it is estimated that about 1.5% of the population experiences tone-deafness, while many others live through various types of perceptual impairments that alter the ways they interact with the world. The condition, also known as congenital amusia, leads to difficulty perceiving pitch differences and generating hedonic responses, ultimately affecting their interconnection with the social and cultural significance of music.
ENSEMBLE is a wearable device that employs sensory substitution technology. By translating sound and music into visual and haptic feedback, it creates a new language of perception that opens the door to new forms of communication and expression. ENSEMBLE allows the users to interact with the surrounding soundscape, amplifying and manipulating their awareness of their immediate environment. It is designed to enable new sensory explorations and foster greater understanding and empathy toward those who experience the world differently.
ENSEMBLE is also a powerful tool for interrogation and introspection. By enveloping the users in a symphony of sensations, it challenges the traditional hierarchy of senses. In doing so, it sheds light on the complexity and diversity of our perceptual realities, and invites us to reflect on our relationship with the world and the role of our senses in shaping it.
PEOPLE DON’T EXPERIENCE SENSES IN THE SAME WAY
AMUSIA
SENSORY SUBSTITUTION
ENSEMBLE
WHAT HAVE I MISSED?
ENSEMBLE translates sound and music into visual and haptic feedback, enabling the users to interact with their surrounding soundscape. It amplifies and manipulates users’ awareness of their immediate environment, creating a new language of perception and communication. This mode opens up new avenues for sensory exploration and, as a result, cultivates empathy toward individuals who experience the world differently.
THE RIGHT TO NOT LISTEN
ENSEMBLE also acts as a new medium for self-expression. This mode becomes activated when the EEG measurement is enabled, and the signal instantly triggers changes in the colors of the light. ENSEMBLE represents the resistance to conforming to social norms and reflects on the amplified collective emotions. It challenges the established hierarchy of senses and interrogates our relationship with the world and the role of our senses in shaping it.
