A study on trust region update rules in Newton methods for large-scale linear classification
Journal
Journal of Machine Learning Research
Journal Volume
77
Pages
33-48
Date Issued
2017
Author(s)
Abstract
The main task in training a linear classifier is to solve an unconstrained minimization problem. To apply an optimization method typically we iteratively find a good direction and then decide a suitable step size. Past developments of extending optimization methods for large-scale linear classification focus on finding the direction, but little attention has been paid on adjusting the step size. In this work, we explain that inappropriate step-size adjustment may lead to serious slow convergence. Among the two major methods for step-size selection, line search and trust region, we focus on investigating the trust region methods. After presenting some detailed analysis, we develop novel and effective techniques to adjust the trust-region size. Experiments indicate that our new settings significantly outperform existing implementations for large-scale linear classification. ? 2017 C.-Y. Hsia, Y. Zhu & C.-J. Lin.
Subjects
Ion beams; Machine learning; Newton-Raphson method; Line searches; Linear classification; Linear classifiers; Optimization method; Step size selection; Trust region; Trust-region methods; Unconstrained minimization problem; Optimization
Type
conference paper