Brain-Aligned Neural Architectures for Real-Time Sound Recognition

SONATA (Sound-Oriented Neural-AI Alignment for Temporal Audition) addresses a core scientific and technological challenge: artificial systems still lack the human brain’s ability to interpret sounds quickly and flexibly. While the brain extracts meaning from sound through dynamic, multiscale processing, current AI systems are trained on static datasets and often fail in unpredictable or noisy conditions. SONATA proposes a new strategy: using high-resolution brain data to guide the development of deep learning models for sound recognition. Specifically, it develops biologically inspired multistream neural networks and constrains their architecture using recordings from magnetoencephalography (MEG), a non-invasive technique that captures brain activity with millisecond precision. The key innovation lies in integrating tools from cognitive neuroscience—such as Representational Similarity Analysis—into the training and evaluation pipeline. These techniques allow the project to align internal model representations with those observed in the brain. The action is structured around three objectives: (O1) design of neuro-inspired model architectures; (O2) incorporation of both theoretical and data-driven neural constraints; and (O3) dual benchmarking of functional accuracy and biological plausibility. SONATA is hosted at Aix-Marseille Université under the supervision of Dr. Bruno Giordano, and will generate open-source tools and insights with applications in auditory neuroscience, AI, and assistive hearing technologies.