HSU WINSTONKennedy, Lyndon S.Lyndon S.KennedyWINSTON HSU2019-07-102019-07-102006-12-011595934472https://scholars.lib.ntu.edu.tw/handle/123456789/413056We propose a novel and generic video/image reranking algorithm, IB reranking, which reorders results from text-only searches by discovering the salient visual patterns of relevant and irrelevant shots from the approximate relevance provided by text results. The IB reranking method, based on a rigorous Information Bottleneck (IB) principle, finds the optimal clustering of images that preserves the maximal mutual information between the search relevance and the high-dimensional low-level visual features of the images in the text search results. Evaluating the approach on the TRECVID 2003-2005 data sets shows significant improvement upon the text search baseline, with relative increases in average performance of up to 23%. The method requires no image search examples from the user, but is competitive with other state-of-the-art example-based approaches. The method is also highly generic and performs comparably with sophisticated models which are highly tuned for specific classes of queries, such as named-persons. Our experimental analysis has also confirmed the proposed reranking method works well when there exist sufficient recurrent visual patterns in the search results, as often the case in multi-source news videos. Copyright 2006 ACM.Information bottleneck principle | Multimodal fusion | Video search[SDGs]SDG4conference paper10.1145/1180639.11806542-s2.0-34547226504https://api.elsevier.com/content/abstract/scopus_id/34547226504