A latent semantic retrieval and clustering system for personal photos with sparse speech annotation
Journal
3rd Workshop on Searching Spontaneous Conversational Speech
Pages
39-40
ISBN
9781605587622
Date Issued
2009
Author(s)
Abstract
In this demo we present a user-friendly latent semantic retrieval and clustering system for personal photos with sparse spontaneous speech tags annotated when the photos were taken. Only 10% of the photos need to be annotated by spontaneous speech of a few words regarding one or two semantic categories (e.g. what or where), while all photos can be effectively retrieved using high-level semantic queries in words (e.g. who, what, where, when) and clustered by the semantics as well. We use low-level image features to construct the relationships among photos, but train semantic models using Probabilistic Latent Semantic Analysis (PLSA) based on fused speech and image features to derive the "topics" of the photos. The sparse speech annotations serve as the user interface for the whole personal photo archive, while photos not annotated are automatically related by fused features and semantic topics of PLSA.
Subjects
Clustering; Fused speech and image features; Photo retrieval; Probabilistic latent semantic analysis (PLSA)
SDGs
Other Subjects
Clustering system; High level semantics; Image features; Latent semantics; Low-level image features; Photo retrieval; Probabilistic latent semantic analysis; Probabilistic latent semantic analysis (PLSA); Semantic category; Semantic Model; Spontaneous speech; Image retrieval; Marine signal systems; Method of moments; Multimedia systems; User interfaces; Semantics
Type
conference paper
File(s)![Thumbnail Image]()
Loading...
Name
d98922032a.pdf
Size
2.27 MB
Format
Adobe PDF
Checksum
(MD5):a0387b1bb60a78fa131c868bd63eaefd