Effects of feature selection methods in estimating SO2 concentration variations using machine learning and stacking ensemble approach
Journal
Environmental Technology and Innovation
Journal Volume
37
Start Page
103996
ISSN
2352-1864
Date Issued
2025-02
Author(s)
Pei-Yi Wong
Yu-Ting Zeng
Huey-Jen Su
Shih-Chun Candice Lung
Yu-Cheng Chen
Pau-Chung Chen
Gary Adamkiewicz
Chih-Da Wu
Abstract
Statistical-based feature selection methods have been used for dimension reduction, but only a few studies have explored the impact of selected features on machine learning models. This study aims to investigate the effects of statistical and machine learning-based feature selection methods on spatial prediction models for estimating variations in SO2 concentrations. We collected daily SO2 observations from 1994 to 2018 along with predictor variables such as land-use/land cover allocations, roads, landmarks, meteorological factors, and satellite images, resulting in a total of 428 geographic predictors. Important features were identified using statistical-based feature selection methods including SelectKBest, stepwise feature selection, elastic net, and machine learning-based methods such as random forest. The selected features from the four feature selection methods were fitted to machine learning algorithms including gradient boosting, CatBoost, XGBoost, and stacking ensemble to establish prediction models for estimating SO2 concentrations. SHapley Additive exPlanations (SHAP) was applied to explain the contribution of each selected feature to the model's prediction capability. The results showed that stacking ensemble model outperformed the three single machine learning algorithms. Among the four feature selection methods, the random forest method yielded the highest prediction accuracy (R2=0.80) in the training model, followed by stepwise selection (R2=0.75), SelectKBest (R2=0.75), and elastic net (R2=0.72) in the stacking ensemble model. These results were robust after several validation tests. Our findings suggested that the random forest feature selection method was more suitable for developing machine learning models for air pollution estimation. The identified features also provide important information for urban air pollution management.
Subjects
Feature selection
Random forest
SO2
Spatial modelling
Stacking ensemble
Publisher
Elsevier BV
Type
journal article