Dual coordinate descent methods for logistic regression and maximum entropy models
Journal
Machine Learning
Journal Volume
85
Journal Issue
1-2
Pages
41-75
Date Issued
2011
Author(s)
Abstract
Most optimization methods for logistic regression or maximum entropy solve the primal problem. They range from iterative scaling, coordinate descent, quasi-Newton, and truncated Newton. Less efforts have been made to solve the dual problem. In contrast, for linear support vector machines (SVM), methods have been shown to be very effective for solving the dual problem. In this paper, we apply coordinate descent methods to solve the dual form of logistic regression and maximum entropy. Interestingly, many details are different from the situation in linear SVM. We carefully study the theoretical convergence as well as numerical issues. The proposed method is shown to be faster than most state of the art methods for training logistic regression and maximum entropy. © 2010 The Author(s).
Type
journal article