Limited-memory common-directions method for distributed L1-regularized linear classification
Journal
SIAM International Conference on Data Mining, SDM 2018
Pages
504-512
Date Issued
2018
Author(s)
Abstract
For distributed linear classification, L1 regularization is useful because of a smaller model size. However, with the non-differentiability, it is more difficult to develop efficient optimization algorithms. In the past decade, OWLQN has emerged as the major method for distributed training of L1 problems. In this work, we point out issues in OWLQN's search directions. Then we extend the recently developed limited-memory common-directions method for L2-regularized problems to L1 scenarios. Through a unified interpretation of batch methods for L1 problems, we explain why OWLQN has been a popular method and why our method is superior in distributed environments. Experiments confirm that the proposed method is faster than OWLQN in most situations. ? 2018 by SIAM.
Subjects
Batch methods; Distributed environments; L1 regularizations; Limited memory; Linear classification; Non-differentiability; Optimization algorithms; Search direction; Data mining
Type
conference paper