Compute the term contributed frequency
Journal
Proceedings - 8th International Conference on Intelligent Systems Design and Applications, ISDA 2008
Journal Volume
2
Pages
325-328
Date Issued
2008
Author(s)
Abstract
In this paper, we propose an algorithm and data structure for computing the term contributed frequency (tcf) for all N-grams in a text corpus. Although term frequency is one of the standard notions of frequency in Corpus-Based Natural Language Processing (NLP), there are some problems regarding the use of the concept to N-grams approaches such as the distortion of phrase frequencies. We attempt to overcome this drawback by building a DAG containing the proposed data structure and using it to retrieve more reliable term frequencies. Our proposed algorithm and data structure are more efficient than traditional term frequency extraction approaches and portable to various languages.
SDGs
Type
conference paper