TPC-NAS: Simple and Effective Neural Architecture Search Based on Total Path Count
Part Of
2024 IEEE 6th International Conference on AI Circuits and Systems, AICAS 2024 - Proceedings
Journal Volume
33
Start Page
542
End Page
546
ISBN (of the container)
979-835038363-8
Date Issued
2024-04-22
Author(s)
Abstract
With the explosive growth of neural network (NN) research and application areas, there is a pressing need to automate the NN model search process in order to attain optimal performance. Nevertheless, existing neural architecture search (NAS) algorithms are time-consuming, resource-intensive, and predominantly tailored for image-related applications. This paper presents the Total Path Count (TPC) score, a straightforward yet highly efficient accuracy predictor solely reliant on the architectural information of a model. The effectiveness of the TPC score is underscored by a robust rank correlation of 0.96 between TPC scores and the accuracies of CIFAR100 architectures. We further introduce TPC-NAS, a zero-shot NAS method that can complete a NAS task in under five CPU minutes without training and inference. TPC-NAS has found wide-ranging applications and it outperforms many other NAS solutions. In image classification, TPC-NAS achieves 78.3% ImageNet top-1 accuracy with 399M FLOPs, while in object detection, it improves mAP by at least 2% over other NAS-derived models. Moreover, TPC-NAS successfully discovers a super-resolution architecture with < 300K parameters and achieves 32.09dB PSNR. In NLP, TPC-NAS delivers a model that matches tinyBERT’s FLOPs but outperforms it by almost 10% in accuracy. These experiments illustrate TPC-NAS’s ability to rapidly generate high-performance CNN/transformer architectures for various applications.
Event(s)
6th IEEE International Conference on AI Circuits and Systems, AICAS 2024
Publisher
IEEE
Type
conference paper