Configuration Tuning on Hadoop System Based on Machine Learning
Date Issued
2014
Date
2014
Author(s)
Chen, Chi-Ou
Abstract
Big Data has emerged in recent year. Systems which is able to support such large-scale data analysis are received more attentions. The distributed system like Hadoop is most used for the analysis. However, it will be increasingly difficult for system administrators to manage the whole system when the cluster of the system scales out. System administrator should maintain the system to execute applications stably. Besides, they need to optimize the system to improve the performance, increase the system utilization and reduce the latency of application executing. And the configuration problem is the most important issue of system optimization. Configuration parameter tuning is related lots of complicated issues. It needs to understand the interaction between physical machines and the behavior of each applications. The current method, rule-based and cost-based optimization, have drawbacks like unfeasibility and limitation of configuration parameter space. Our work exploit machine learning to solve the problem to improve the performance.
Subjects
巨量資料
分散式系統
機器學習
全局優化
隨機抽樣
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-103-R01922108-1.pdf
Size
23.32 KB
Format
Adobe PDF
Checksum
(MD5):a70ff68b7ef50bbfd8477b96fdf7b2f1