FQ4DM: Full Quantization for Diffusion Model
Part Of
IEEE International Workshop on Machine Learning for Signal Processing, MLSP
Start Page
1
End Page
6
ISBN (of the container)
979-835037225-0
Date Issued
2024-09-22
Author(s)
Abstract
Diffusion models (DMs) have recently gained acclaim for their superior imaging capabilities. However, their extensive computational and memory demands often limit the practical application on portable devices. Post-training quantization (PTQ) offers a solution that enables model compression and reduces runtime without retraining. Nonetheless, traditional PTQ methods struggle to handle the unique time-variant distribution in DMs. Accordingly, we propose a novel timestep-grouping PTQ approach to address the multiple timestep issue. We also identify that non-uniform post-SiLU activations may lead to significant quantization loss. We tackle this issue with a region-specific quantization strategy that better represents extreme values after quantization. Combined with the above methods, we achieve a fully quantized diffusion model feasible for hardware implementation. Our experimental results show that the proposed method successfully maintains the FID score after 8-bit quantization.
Event(s)
34th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2024
Publisher
IEEE
Type
conference paper