https://scholars.lib.ntu.edu.tw/handle/123456789/632500
標題: | Cold-start active learning through self-supervised language modeling | 作者: | Yuan M HSUAN-TIEN LIN Boyd-Graber J. |
公開日期: | 2020 | 起(迄)頁: | 7935-7948 | 來源出版物: | EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference | 摘要: | Active learning strives to reduce annotation costs by choosing the most critical examples to label. Typically, the active learning strategy is contingent on the classification model. For instance, uncertainty sampling depends on poorly calibrated model confidence scores. In the cold-start setting, active learning is impractical because of model instability and data scarcity. Fortunately, modern NLP provides an additional source of information: pretrained language models. The pre-training loss can find examples that surprise the model and should be labeled for efficient fine-tuning. Therefore, we treat the language modeling loss as a proxy for classification uncertainty. With BERT, we develop a simple strategy based on the masked language modeling loss that minimizes labeling costs for text classification. Compared to other baselines, our approach reaches higher accuracy within less sampling iterations and computation time. © 2020 Association for Computational Linguistics. |
URI: | https://www.scopus.com/inward/record.uri?eid=2-s2.0-85110551263&partnerID=40&md5=005bd5f7a39b83e8a198b3dae6887fc2 https://scholars.lib.ntu.edu.tw/handle/123456789/632500 |
SDG/關鍵字: | Artificial intelligence; Classification (of information); Computational linguistics; Learning systems; Modeling languages; Natural language processing systems; Uncertainty analysis; Active Learning; Active learning strategies; Calibrated model; Classification models; Cold-start; Confidence score; Data scarcity; Language model; Sources of informations; Uncertainty samplings; Text processing |
顯示於: | 資訊工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。