Using Language to Learn Structured Appearance Models for Image Annotation

Jamieson M, Fazly A, Stevenson S, Dickinson S, Wachsmuth S (2010)
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 32(1): 148-164.

Journal Article | Published | English

No fulltext has been uploaded

Author
; ; ; ;
Abstract
Given an unstructured collection of captioned images of cluttered scenes featuring a variety of objects, our goal is to simultaneously learn the names and appearances of the objects. Only a small fraction of local features within any given image are associated with a particular caption word, and captions may contain irrelevant words not associated with any image object. We propose a novel algorithm that uses the repetition of feature neighborhoods across training images and a measure of correspondence with caption words to learn meaningful feature configurations (representing named objects). We also introduce a graph-based appearance model that captures some of the structure of an object by encoding the spatial relationships among the local visual features. In an iterative procedure, we use language (the words) to drive a perceptual grouping process that assembles an appearance model for a named object. Results of applying our method to three data sets in a variety of conditions demonstrate that, from complex, cluttered, real-world scenes with noisy captions, we can learn both the names and appearances of objects, resulting in a set of models invariant to translation, scale, orientation, occlusion, and minor changes in viewpoint or articulation. These named models, in turn, are used to automatically annotate new, uncaptioned images, thereby facilitating keyword-based image retrieval.
Publishing Year
ISSN
PUB-ID

Cite this

Jamieson M, Fazly A, Stevenson S, Dickinson S, Wachsmuth S. Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. 2010;32(1):148-164.
Jamieson, M., Fazly, A., Stevenson, S., Dickinson, S., & Wachsmuth, S. (2010). Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 32(1), 148-164.
Jamieson, M., Fazly, A., Stevenson, S., Dickinson, S., and Wachsmuth, S. (2010). Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 32, 148-164.
Jamieson, M., et al., 2010. Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 32(1), p 148-164.
M. Jamieson, et al., “Using Language to Learn Structured Appearance Models for Image Annotation”, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 32, 2010, pp. 148-164.
Jamieson, M., Fazly, A., Stevenson, S., Dickinson, S., Wachsmuth, S.: Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. 32, 148-164 (2010).
Jamieson, Michael, Fazly, Afsaneh, Stevenson, Suzanne, Dickinson, Sven, and Wachsmuth, Sven. “Using Language to Learn Structured Appearance Models for Image Annotation”. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 32.1 (2010): 148-164.
This data publication is cited in the following publications:
This publication cites the following data publications:

Export

0 Marked Publications

Open Data PUB

Web of Science

View record in Web of Science®

Sources

PMID: 19926905
PubMed | Europe PMC

Search this title in

Google Scholar