How time matters: Learning time-decay attention for contextual spoken language understanding in dialogues
Journal
NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference
Journal Volume
1
Pages
2133-2142
Date Issued
2018
Author(s)
Abstract
Spoken language understanding (SLU) is an essential component in conversational systems. Most SLU components treat each utterance independently, and then the following components aggregate the multi-Turn information in the separate phases. In order to avoid error propagation and effectively utilize contexts, prior work leveraged history for contextual SLU. However, most previous models only paid attention to the related content in history utterances, ignoring their temporal information. In the dialogues, it is intuitive that the most recent utterances are more important than the least recent ones, in other words, timeaware attention should be in a decaying manner. Therefore, this paper designs and investigates various types of time-decay attention on the sentence-level and speaker-level, and further proposes a flexible universal time-decay attention mechanism. The experiments on the benchmark Dialogue State Tracking Challenge (DSTC4) dataset show that the proposed time-decay attention mechanisms significantly improve the state-of-The-Art model for contextual understanding performance1. ? 2018 The Association for Computational Linguistics.
Subjects
Computational linguistics; Attention mechanisms; Contextual understanding; Conversational systems; Error propagation; Related content; Spoken language understanding; State of the art; Temporal information; Decay (organic)
Type
conference paper