Less is More: Filtering Abnormal Dimensions in GloVe.
Journal
Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11-15, 2016, Companion Volume
Pages
71-72
Date Issued
2016
Author(s)
Abstract
GloVe, global vectors for word representation, performs well in some word analogy and semantic relatedness tasks. However, we find that some dimensions of the trained word embedding are abnormal. We verify our conjecture via removing these abnormal dimensions using Kolmogorov-Smimov test and experiment on several benchmark datasets for semantic relatedness measurement. The experimental results confirm our finding. Interestingly, some of the tasks outperform the state-of-the-art model SensEmbed by simply removing these abnormal dimensions. The novel rule of thumb technique which leads to better performance is expected to be useful in practice. © 2016 owner/author(s).
Subjects
glove; semantic relatedness; word embedding
Other Subjects
Embeddings; ART model; Benchmark datasets; Embeddings; Glove; Kolmogorov; Less is mores; Semantic relatedness; State of the art; Word embedding; Word representations; Semantics
Type
conference paper