4D C-String: A New Audio-visual Knowledge Structure and Similarity Retrieval for Video Database Systems

This paper presents a new audio-visual knowledge structure and similarity for video database systems, called 4D C-string. It is based on the 3D C-string, which is a knowledge structure that can express visual characteristic of objects in a video but it does not consider the audio part of videos. So we add audio dimension on it to make the retrieval results more precise. For the visual part, we can generate strings to represent the spatial and temporal relations between the objects in a video and their motions and size changes. For the audio part, we can generate three audio strings. Then we propose the similarity retrieval algorithm based on the visual and audio information to retrieve the similar videos from the database for a given query video. Our proposed method this approach can provide user an easy and efficient way to retrieve, visualize and manipulate video and audio objects in video database systems.