https://scholars.lib.ntu.edu.tw/handle/123456789/581294
標題: | A Study on Truncated Newton Methods for Linear Classification | 作者: | Galli L Lin C. CHIH-JEN LIN |
關鍵字: | Newton-Raphson method; Convergence theory; Differentiability; Large-scale optimization; Linear classification; Local Convergence; Stopping criteria; Truncated Newton method; Truncated newtons; Conjugate gradient method | 公開日期: | 2021 | 來源出版物: | IEEE Transactions on Neural Networks and Learning Systems | 摘要: | Truncated Newton (TN) methods have been a useful technique for large-scale optimization. Instead of obtaining the full Newton direction, a truncated method approximately solves the Newton equation with an inner conjugate gradient (CG) procedure (TNCG for the whole method). These methods have been employed to efficiently solve linear classification problems. However, even in this deeply studied field, various theoretical and numerical aspects were not completely explored. The first contribution of this work is to comprehensively study the global and local convergence when TNCG is applied to linear classification. Because of the lack of twice differentiability under some losses, many past works cannot be applied here. We prove various missing pieces of theory from scratch and clarify many proper references. The second contribution is to study the termination of the CG method. For the first time when TNCG is applied to linear classification, we show that the inner stopping condition strongly affects the convergence speed. We propose using a quadratic stopping criterion to achieve both robustness and efficiency. The third contribution is that of combining the study on inner stopping criteria with that of preconditioning. We discuss how convergence theory is affected by preconditioning and finally propose an effective preconditioned TNCG. IEEE |
URI: | https://www.scopus.com/inward/record.uri?eid=2-s2.0-85099603754&doi=10.1109%2fTNNLS.2020.3045836&partnerID=40&md5=2b62275a74a403a00ef15647b19fa435 https://scholars.lib.ntu.edu.tw/handle/123456789/581294 |
ISSN: | 2162237X | DOI: | 10.1109/TNNLS.2020.3045836 |
顯示於: | 資訊工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。