OmniScribe: Authoring Immersive Audio Descriptions for 360° Videos
Journal
UIST 2022 - Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology
Pages
1-14
ISBN
9781450393201
Date Issued
2022-10-29
Author(s)
Chang, Ruei-Che
Ting, Chao-Hsien
Hung, Chia-Sheng
Chen, Liang-Jin
Chao, Yu-Tzu
Guo, Anhong
Lee, Wan-Chen
Abstract
Blind people typically access videos via audio descriptions (AD) crafted by sighted describers who comprehend, select, and describe crucial visual content in the videos. 360° video is an emerging storytelling medium that enables immersive experiences that people may not possibly reach in everyday life. However, the omnidirectional nature of 360° videos makes it challenging for describers to perceive the holistic visual content and interpret spatial information that is essential to create immersive ADs for blind people. Through a formative study with a professional describer, we identified key challenges in describing 360° videos and iteratively designed OmniScribe, a system that supports the authoring of immersive ADs for 360° videos. OmniScribe uses AI-generated content-awareness overlays for describers to better grasp 360° video content. Furthermore, OmniScribe enables describers to author spatial AD and immersive labels for blind users to consume the videos immersively with our mobile prototype. In a study with 11 professional and novice describers, we demonstrated the value of OmniScribe in the authoring workflow; and a study with 8 blind participants revealed the promise of immersive AD over standard AD for 360° videos. Finally, we discuss the implications of promoting 360° video accessibility.
Subjects
360° video | accessibility | audio description | Blind | computer vision | mobile | multimedia | sonification | virtual reality | visual impairment
SDGs
Type
conference paper