HeteroYarn: A Self-Tuning Resource Management System for Hadoop on Heterogeneous Platforms
Date Issued
2014
Date
2014
Author(s)
Su, Jia-Kuan
Abstract
The booming of Apache Hadoop solves many kinds of big data problems, but the poor performance of Hadoop applications due to the bottlenecks of computing is always reviled. Porting Hadoop applications to accelerators, such as GPUs, is a solution to speedup the performance. However, programmers may take great effort to redesign applications for GPUs, and have troubles with managing the CPU and GPU resources. It is not feasible to let users handle the above difficulties.
In this thesis, We proposed a framework which combines Hadoop YARN and Aparapi library for computing resources management in heterogeneous platforms. We provided an API to help users in porting their MapReduce applications onto heterogeneous platforms. Our work uses an optimized strategy to minimize the execution of a Hadoop application by profiling the execution time of tasks on CPUs and GPUs. We also proposed several methods to fairly share the CPU and GPU resources among running applications in the cluster. In the experiments, we show the speedup of an application, and analyze the effects to performance by different methods for resources fair sharing among multiple applications.
Subjects
巨量資料
異質平台
資源管理
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-103-R01922081-1.pdf
Size
23.32 KB
Format
Adobe PDF
Checksum
(MD5):de3be1372295388c3f1b9938c9bcdc4c