Named entity recognition from spoken documents using global evidences and external knowledge sources with applications on Mandarin Chinese
Journal
Proceedings of ASRU 2005: 2005 IEEE Automatic Speech Recognition and Understanding Workshop
Journal Volume
2005
Pages
390-395
Date Issued
2005
Author(s)
Abstract
In this paper, we propose two efficient approaches for Named Entity recognition (NER) from spoken documents. The first approach used a very efficient data structure, the PAT trees, to extract global evidences from the whole spoken documents, to be used with the well-known local (internal and external) evidences popularly used by conventional approaches. The basic idea is that a Named Entity (NE) may not be easily recognized in certain contexts, but may become much more easily recognized when its repeated occurrences in all the different sentences in the same spoken document are considered jointly. This approach is equally useful for NER from text and spoken documents. The second approach is to try to recover some Named Entities (NEs) which are out-of-vocabulary (OOV) words and thus can't be obtained in the transcriptions. The basic idea is to use reliable and important words in the transcription to construct queries to retrieve relevant text documents from external knowledge sources (such as Internet). Matching the NEs obtained from these retrieved relevant text documents with some selected sections of the phone lattice of the spoken document can recover some NEs which are OOV words. The experiments were performed on Mandarin Chinese by incorporating these two approaches to a conventional hybrid statistic/rule-based NER system for Chinese language. Very significant performance improvements were obtained. © 2005 IEEE.
Event(s)
ASRU 2005: 2005 IEEE Automatic Speech Recognition and Understanding Workshop
SDGs
Other Subjects
Data structures; Information retrieval; Information retrieval systems; Statistical methods; Text processing; Trees (mathematics); External knowledge sources; Named Entity recognition (NER); Speech recognition
Type
conference paper