IMPACT: Iterative Mask-based Parallel Decoding for Text-to-Audio Generation with Diffusion Modeling
Journal
Proceedings of Machine Learning Research
Journal Volume
267
Start Page
26002
End Page
26019
ISSN
26403498
Date Issued
2025-07
Author(s)
Huang, Kuan-Po
Yang, Shu-Wen
Phan, Huy
Lu, Bo-Ru
Kim, Byeonggeun
Macha, Sashank
Tang, Qingming
Ghosh, Shalini
Kao, Chieh-Chi
Wang, Chao
Abstract
Text-to-audio generation synthesizes realistic sounds or music given a natural language prompt. Diffusion-based frameworks, including the Tango and the AudioLDM series, represent the state-ofthe-art in text-to-audio generation. Despite achieving high audio fidelity, they incur significant inference latency due to the slow diffusion sampling process. MAGNET, a mask-based model operating on discrete tokens, addresses slow inference through iterative mask-based parallel decoding. However, its audio quality still lags behind that of diffusion-based models. In this work, we introduce IMPACT, a text-to-audio generation framework that achieves high performance in audio quality and fidelity while ensuring fast inference. IMPACT utilizes iterative mask-based parallel decoding in a continuous latent space powered by diffusion modeling. This approach eliminates the fidelity constraints of discrete tokens while maintaining competitive inference speed. Results on AudioCaps demonstrate that IMPACT achieves state-of-the-art performance on key metrics including Fréchet Distance (FD) and Fréchet Audio Distance (FAD) while significantly reducing latency compared to prior models. The project website is available at https://audio-impact.github.io/.
Event(s)
42nd International Conference on Machine Learning, ICML 2025
Publisher
ML Research Press
Type
conference paper