https://scholars.lib.ntu.edu.tw/handle/123456789/489627
標題: | A Fast and Scalable Cluster Simulator for Network Performance Projection of HPC Applications | 作者: | Liu, C.-Y. Huang, P.-Y. Tu, C.-H. SHIH-HAO HUNG |
關鍵字: | Communication pattern; Computer cluster; Full-system emulation; High-performance computing; Network simulation; Phase-Aware acceleration | 公開日期: | 2018 | 起(迄)頁: | 970-977 | 來源出版物: | Proceedings - 2018 International Conference on High Performance Computing and Simulation, HPCS 2018 | 摘要: | For high-performance computing (HPC) cluster to solve distributed computing problems with big data, the performance is usually bounded by communication ability of each node and the underlying network architecture of the cluster. Network simulations are commonly adopted to evaluate the impacts of network configurations before the deployment of such a high-cost system, and a fast and scalable network performance simulation with profiling mechanism is crucial to explore the design space in time effectively. This work presents Clusim, a cluster simulator for exploration of the performance of interplay among distributed HPC applications and network architectures, where three primary techniques are developed in Clusim·. 1) the full system emulators forming a virtual cluster for collecting the network communication traces, 2) window-based simulation, and 3) phase-Aware accelerated network simulation. The first technique helps to reduce the cost and efforts of establishing a physical cluster system for performance estimation. The second one enables cross-platform performance projection by thoroughly preserving the concurrency of the recorded packet flows during simulation. The last one efficiently reduces the simulation time by skipping the detailed simulations of repeated phases of network traffics. We validate Clusim against a physical cluster and show that the difference of the estimated end-To-end delays of the simulated network and those measured on the physical counterpart is within 15%. Meanwhile, thanks to the phase-Aware simulation scheme, Clusim achieves up to 18x speedups, compared with the packet-by-packet simulation by ns-3. Our results suggest that Clusim is a promising design and can rapidly provide the network performance projection of a large scale system with the appropriate trade-off between accuracy and time. © 2018 IEEE. |
URI: | https://scholars.lib.ntu.edu.tw/handle/123456789/489627 https://www.scopus.com/inward/record.uri?eid=2-s2.0-85057364968&doi=10.1109%2fHPCS.2018.00153&partnerID=40&md5=c877a0b94847d15efb1a43e6b293c33c |
DOI: | 10.1109/HPCS.2018.00153 | SDG/關鍵字: | Big data; Cluster computing; Economic and social effects; Large scale systems; Network performance; Communication pattern; Computer clusters; High performance computing; Network simulation; System emulation; Network architecture |
顯示於: | 資訊工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。