https://scholars.lib.ntu.edu.tw/handle/123456789/581437
標題: | Modeling Interprocessor Communication and Performance Scalability for Distributed Deep Learning Systems | 作者: | Lyu Y.-H Liu C.-Y Lee C.-P Tu C.-H Hung S.-H. SHIH-HAO HUNG |
關鍵字: | Computer hardware; Computer networks; Deep learning; Open source software; Open systems; Scalability; Computer networking; Computing hardware; High-speed interconnects; Inter processor communication; Performance prediction; Performance scalability; Scalability analysis; Underlying systems; Learning systems | 公開日期: | 2019 | 起(迄)頁: | 169-176 | 來源出版物: | 2019 International Conference on High Performance Computing and Simulation, HPCS 2019 | 摘要: | While deep learning applications become popular, the design of deep learning systems is a critical task to unleash the computing power of underlying systems. Aside from the computing hardware, the computer networking is also a key factor that affects the delivered performance. When considering a large and complex model, the scalability of the system highly depends on the design of the networks, as well as the software behaviors. In this paper, we propose a profile-data-guided performance prediction method to estimate the performance of the system with desired high-speed interconnects, based on the profiling data obtained in a previous run. In particular, we leverage the open-source profiling tool, SOFA, for characterizing the software activities of deep learning software running in a computer cluster, and the characterized information is used to build the performance model for the model training process. When estimating the performance, SOFA is used to capture the performance critical factors for the model to make the predictions. To evaluate the proposed method, four popular deep learning models are adopted in our experiments, ResNet50, Inception3, Alexnet, and VGG16, where a computer cluster formed by four nodes is used to profile the training of the above models on TensorFlow. We ran the scalability analysis to analyze the size of the cluster, and the suitable computer networks for the models. By comparing the predicted data and those measured on the cluster, our model achieves up to 95% accuracy in most of the cases, with the maximum error rate of 10%. ? 2019 IEEE. |
URI: | https://www.scopus.com/inward/record.uri?eid=2-s2.0-85092042686&doi=10.1109%2fHPCS48598.2019.9188168&partnerID=40&md5=c0a59ee6c668b9afef2668017e8b17c1 https://scholars.lib.ntu.edu.tw/handle/123456789/581437 |
DOI: | 10.1109/HPCS48598.2019.9188168 |
顯示於: | 資訊工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。