This research explores the interaction of textual and photographic information in an integrated text/image database environment. Specifically, We present a content-based retrieval system for captioned group photographs of people (i.e., human faces) where groups can consist of one or more members. By understanding the caption accompanying a picture, we are able to extract information useful in (i) retrieving the picture and (ii) directing an image interpretation system identify relevant objects (in this case, faces) in the picture. For the latter, we incorporate techniques from our ongoing research on photo understanding using accompanying text. Current image-based techniques have limitations; for example, similarity techniques used for retrieving faces will not perform well in group photographs where the locations of faces is not known a priori or where face sizes are small. By exploiting caption information, we assist a face locator in detecting human faces in a photograph and subsequently labelling them. Text-based similarity algorithms have principally relied on statistical techniques to index and classify documents (e.g., vector models). It is necessary to employ natural language processing techniques in order to derive deeper semantics from captions which contain far fewer words than documents. Our approach is unique since it goes beyond a superficial combination of existing text-based and image-based approaches to information retrieval.
Automatic indexing and content-based retrieval of captioned photographs
Proceedings of 3rd International Conference on Document Analysis and Recognition ; 2 ; 1165-1168 vol.2
1995-01-01
597583 byte
Conference paper
Electronic Resource
English
Automatic Indexing and Content-Based Retrieval of Captioned Photographs
British Library Conference Proceedings | 1995
|ABI: analogy-based indexing for content image retrieval
British Library Online Contents | 2004
|Viewpoint-Invariant Indexing for Content-Based Image Retrieval
British Library Conference Proceedings | 1998
|Content-based image retrieval by viewpoint-invariant color indexing
British Library Online Contents | 1999
|