https://scholars.lib.ntu.edu.tw/handle/123456789/608044
Title: | Nonparametric Regression via Variance-Adjusted Gradient Boosting Gaussian Process Regression | Authors: | HSIN-MIN LU Chen J.-S Liao W.-C. |
Keywords: | big data;Gaussian process;Gradient boosting;variance-adjusted prediction;Covariance matrix;Data Analytics;Decision trees;Gaussian noise (electronic);Large dataset;Stochastic models;Stochastic systems;Support vector machines;Support vector regression;Broad application;Gaussian process regression;Gaussian process regression model;Non-parametric regression;Prediction performance;Training subsets;Variational inference;Gaussian distribution | Issue Date: | 2021 | Journal Volume: | 33 | Journal Issue: | 6 | Start page/Pages: | 2669-2679 | Source: | IEEE Transactions on Knowledge and Data Engineering | Abstract: | Regression models have broad applications in data analytics. Gaussian process regression is a nonparametric regression model that learns nonlinear maps from input features to real-valued output using a kernel function that constructs the covariance matrix among all pairs of data. Gaussian process regression often performs well in various applications. However, the time complexity of Gaussian process regression is O(n3) O(n3) for a training dataset of size $n$ n. The cubic time complexity hinders Gaussian process regression from scaling up to large datasets. Guided by the properties of Gaussian distributions, we developed a variance-adjusted gradient boosting algorithm for approximating a Gaussian process regression (VAGR). VAGR sequentially approximates the full Gaussian process regression model using the residuals computed from variance-adjusted predictions based on randomly sampled training subsets. VAGR has a time complexity of O(nm3) O(nm3) for a training dataset of size n n and the chosen batch size $m$ m. The reduced time complexity allows us to apply VAGR to much larger datasets compared with the full Gaussian process regression. Our experiments suggest that VAGR has a prediction performance comparable to or better than models that include random forest, gradient boosting machines, support vector regressions, and stochastic variational inference for Gaussian process regression. ? 1989-2012 IEEE. |
URI: | https://www.scopus.com/inward/record.uri?eid=2-s2.0-85098927817&doi=10.1109%2fTKDE.2019.2953728&partnerID=40&md5=37bf161cc567dc630069097c73d518c3 https://scholars.lib.ntu.edu.tw/handle/123456789/608044 |
ISSN: | 10414347 | DOI: | 10.1109/TKDE.2019.2953728 |
Appears in Collections: | 資訊管理學系 |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.