AMP-BiLSTM: An Enhanced Highlight Extraction Method Using Multi-Channel Bi-LSTM and Self-Attention in Streaming Videos
Journal
2024 IEEE 18th International Conference on Semantic Computing (ICSC)
Start Page
9
End Page
16
ISSN
2325-6516
2472-9671
ISBN
9798350385359
Date Issued
2024-02-05
Author(s)
DOI
10.1109/ICSC59802.2024.00009
Abstract
With the rise of conversation-oriented streaming videos, the platforms that host them like Twitch have rapidly become prominent information hubs. However, the lengthy nature of such streams often deters viewers from consuming the full content. To mitigate this, we propose AMP-BiLSTM, a novel highlight extraction method which focuses on the textual information in streamer discourses and viewer responses rather than visual features. This approach addresses the limitations of previous methods, which primarily centered on analyzing visual features, and were thus insufficient for conversation-oriented streams where highlights emerge from dialogues and viewer interactions. AMP-BiLSTM is built on techniques of Attention, Multi-channel, and Position enrichment integrated into a Bidirectional Long Short-Term Memory (BiLSTM) network. Through experiments on a real-world dataset, we found that streamer discourses and viewer messages provide significant utility for highlight extraction in conversation-oriented streaming videos. Furthermore, our proposed Multi-channel and self-attention techniques effectively distill text streams into semantically-rich embeddings. The experiment results demonstrate that AMP-BiLSTM outperforms several state-of-the-art methods for deep learning-based highlight extraction, thus showing promise for improved conversation-oriented streaming video content digestion.
Event(s)
18th IEEE International Conference on Semantic Computing, ICSC 2024
Subjects
Attention Mechanism
BiLSTM
Conversation-Oriented Streaming Video
Deep Learning
Highlight Extraction
Multi-channel Analysis
Position Enrichment
Publisher
IEEE
Description
18th IEEE International Conference on Semantic Computing, ICSC 2024 - Hybrid, Laguna Hills, United States
Duration: Feb 5 2024 → Feb 7 2024
Duration: Feb 5 2024 → Feb 7 2024
Type
conference proceedings