Analyzing the Interplay between Random Shuffling and Storage Devices for Efficient Machine Learning
Journal
Proceedings - 2021 IEEE International Symposium on Performance Analysis of Systems and Software, ISPASS 2021
Pages
276-287
Date Issued
2021
Author(s)
Abstract
Machine learning algorithms, such as Support Vector Machine (SVM) and Deep Neural Network (DNN), have gained a lot of interest recently. When training a machine learning algorithm, randomly shuffling all the training data can improve the testing accuracy and boost the convergence rate. Nevertheless, realizing training data random shuffling in a real system is not straightforward due to the slow random accesses in hard disk drives (HDDs). Common random shuffling implementations assume that HDD is used as storage, so they sacrifice the random degree of shuffling to reduce random storage accesses. Different from conventional HDD, emerging solid-state drive (SSD) based storage devices, such as Intel Optane SSD, offer fast random accesses. In this paper, we explore the opportunities to take advantage of the fast random access property in SSD to perform full-range random shuffling without taking up precious CPU memory and study the interplay between different shuffling methods and various types of storage devices. We use a lightweight implementation of random shuffling (LIRS) as an example of the SSD-aware shuffling method to conduct performance analysis. Evaluations show that, compared to conventional shuffling methods, LIRS can improve convergence rate and reduce the total training time of SVM and DNN by 67.1% and 33.9% on average. ? 2021 IEEE.
Subjects
Deep learning; Deep neural networks; Hard disk storage; Learning systems; Random access storage; Support vector machines; Turing machines; Virtual storage; Convergence rates; Hard disk drives; Performance analysis; Random access; Random storage; Solid state drives (SSD); Testing accuracy; Training time; Learning algorithms
SDGs
Type
conference paper