An Image Retrieval System Using Music as Query
Date Issued
2009
Date
2009
Author(s)
Hsu, Chao-Liang
Abstract
In this paper, a novel image retrieval approach is proposed. Differ from traditional image retrieval approaches, which generally retrieve images using keywords or example images as query, the image retrieval system proposed allows the user to search images using music as query. Namely, a music-image cross-media retrieval system is developed. There is rich textual information associated with music and image on the web, and the textual information is used to bridge the semantic gap between music and image in our research. The relevance of music and image are measured by a ranking function derived from Okapi BM25. Music-image semantic matrix is constructed based-on textual information of music and image, and PLSA (Probabilistic Latent Semantic Analysis) is applied on it to measure HSF (hidden semantic feature) of music and image. Neural Network is used to train a mapping function from music audio feature to HSF. In the phase of image retrieval, the music-image retrieval is based on HSF and textual feature. Finally, user relevance feedback is used for image reranking (short-term learning) and updating the music-image descriptive word map (long-term learning) to enhance the retrieval results. To evaluate the image retrieval system, 4000 images with textual information (metadata) are collected from Flickr, 1836 songs are collected and textual information (metadata) of these songs are collected from AMG(All Music Guide). The results show that this image retrieval system can achieve good performance.
Subjects
Image retrieval
cross-media retrievallmetadata
search
relevance feedback
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-98-R96921033-1.pdf
Size
23.32 KB
Format
Adobe PDF
Checksum
(MD5):3ea11569288c3cb0ae12434a10194b64