Chang, Cheng YangCheng YangChangHuang, Chi TseChi TseHuangYU-CHUAN CHUANGChou, Kuang ChaoKuang ChaoChouAN-YEU(ANDY) WU2024-04-242024-04-242024-01-019798350393545https://scholars.lib.ntu.edu.tw/handle/123456789/642024Convolutional neural networks (CNNs) are known for their exceptional performance in various applications; however, their energy consumption during inference can be substantial. Analog Computing-In-Memory (CIM) has shown promise in enhancing the energy efficiency of CNNs, but the use of analog-to-digital converters (ADCs) remains a challenge. ADCs convert analog partial sums from CIM crossbar arrays to digital values, with high-precision ADCs accounting for over 60% of the system's energy. Researchers have explored quantizing CNNs to use low-precision ADCs to tackle this issue, trading off accuracy for efficiency. However, these methods necessitate data-dependent adjustments to minimize accuracy loss. Instead, we observe that the first most significant toggled bit indicates the optimal quantization range for each input value. Accordingly, we propose a range-aware rounding (RAR) for runtime bit-width adjustment, eliminating the need for pre-deployment efforts. RAR can be easily integrated into a CIM accelerator using dynamic block-floating-point arithmetic. Experimental results show that our methods maintain accuracy while achieving up to 1.81 × and 2.08 × energy efficiency improvements on CIFAR-10 and ImageNet datasets, respectively, compared with state-of-the-art techniques.[SDGs]SDG7conference paper10.1109/ASP-DAC58780.2024.104737972-s2.0-85189320726https://api.elsevier.com/content/abstract/scopus_id/85189320726