Full-Duplex-Bench: A Benchmark to Evaluate Full-Duplex Spoken Dialogue Models on Turn-taking Capabilities
Journal
ASRU 2025 - 2025 IEEE Automatic Speech Recognition and Understanding Workshop
Start Page
1
End Page
8
ISBN (of the container)
979-833154426-3
ISBN
[9798331544263]
Date Issued
2025-12-06
Author(s)
Abstract
Spoken dialogue modeling poses challenges beyond text-based language modeling, requiring real-time interaction, turn-taking, and backchanneling. While most Spoken Dialogue Models (SDMs) operate in half-duplex mode - processing one turn at a time - emerging full-duplex SDMs can listen and speak simultaneously, enabling more natural conversations. However, current evaluations remain limited, focusing mainly on turn-based metrics or coarse corpus-level analyses. To address this, we introduce Full-Duplex-Bench, a benchmark that systematically evaluates key interactive behaviors: pause handling, backchanneling, turn-taking, and interruption management. Our framework uses automatic metrics for consistent, reproducible assessment and provides a fair, fast evaluation setup. By releasing our benchmark and code, we aim to advance spoken dialogue modeling and foster the development of more natural and engaging SDMs.
Event(s)
2025 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2025
Subjects
Benchmark
Full-Duplex Dialogue
Spoken Language Models
Turn-Taking
Publisher
Institute of Electrical and Electronics Engineers(IEEE)
Type
conference paper