Efficient input and output for scientific simulations
Journal
Proceedings of the Annual Workshop on I/O in Parallel and Distributed Systems, IOPADS
Pages
33-44
Date Issued
1999
Author(s)
Abstract
Large simulations which run for hundreds of hours on parallel computers often periodically generate snapshots of states, which are later post-processed to visualize the simulated physical phenomenon. For many applications, fast I/O during post-processing, which is dependent on an efficient organization of data on disk, is as important as minimizing computation-time I/O. In this paper we propose optimizations to support efficient parallel I/O for scientific simulations and subsequent visualizations. We present an ordering mechanism to linearize data on disk, a performance model to help to choose a proper stripe unit size, and a scheduling algorithm to minimize communication contention. Our experiments on an IBM SP show that the combination of these strategies provides a 20-25% performance boost.
Other Subjects
Algorithms; Computer simulation; File organization; Input output programs; Mathematical models; Natural sciences computing; Optimization; Data organization; Parallel processing systems
Type
journal article