Identifying Speaker Information in Feed-Forward Layers of Self-Supervised Speech Transformers
Journal
2025 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2025
Start Page
525
End Page
530
ISBN (of the container)
979-833157206-8
Date Issued
2025-11-28
Author(s)
Abstract
In recent years, the impact of self-supervised speech Transformers has extended to speaker-related applications. However, little research has explored how these models encode speaker information. In this work, we address this gap by identifying neurons in the feed-forward layers that are correlated with speaker information. Specifically, we analyze neurons associated with k-means clusters of self-supervised features and i-vectors. Our analysis reveals that these clusters correspond to broad phonetic and gender classes, making them suitable for identifying neurons that represent speakers. By protecting these neurons during pruning, we can significantly preserve performance on speaker-related task, demonstrating their crucial role in encoding speaker information.
Event(s)
17th Asia Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2025
Publisher
IEEE
Type
conference paper