Example-based Object Retrieval in Large-Scale User-Contributed Photos

Due to the exponential growth of image collections, there arise the needs for efficient and effective example-based image search. Recently, image search by matching bags of feature points or visual words has shown an important paradigm, where the search quality is assured by the representation of objects in photos and spatial verification. In this thesis, we aim to investigate two important topics in example-based object retrieval. The first one is efficient spatial verification, which exploits geometry model between matching candidates for rejecting false positives and entailing further applications. The traditional methods for spatial verification are time-consuming to estimate model parameters iteratively from a set of (noisy) observed data. Instead, we observe that the image matching for large-scale image retrieval often corresponds to certain regions in the query image – the hot spots. Therefore, the aim of the proposed novel approach – Locality Sensitive Sample Consensus (LOCSAC), attempts to explore ”good matches” for accurate geometry model estimation. In addition, an online framework is devised for adaptively updating hot spot regions. The second one is the representation of objects in photos. Due to a database image representation generally carries mixed information of the entire image which may contain multiple objects and background. To tackle this problem, we propose a novel representation of objects, pseudo-objects – a subset of proximate feature points with its own feature vector to represent a local area, to approximate candidate objects in database images. Experimenting over consumer photo benchmarks, we will show that the proposed spatial verification method can bring (on the average) 20 folds speed-up over the conventional methods and assure the same or better quality. Besides, we confirm that the proposed pseudo-object can significantly benefit for object retrieval both in accuracy and efficiency.