Using Language to Learn Structured Appearance Models for Image Annotation
Jamieson M, Fazly A, Stevenson S, Dickinson S, Wachsmuth S (2010)
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 32(1): 148-164.
Zeitschriftenaufsatz
| Veröffentlicht | Englisch
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
Autor*in
Jamieson, Michael;
Fazly, Afsaneh;
Stevenson, Suzanne;
Dickinson, Sven;
Wachsmuth, SvenUniBi ![](https://pub.uni-bielefeld.de/images/icon_orcid.png)
![](https://pub.uni-bielefeld.de/images/icon_orcid.png)
Einrichtung
Abstract / Bemerkung
Given an unstructured collection of captioned images of cluttered scenes featuring a variety of objects, our goal is to simultaneously learn the names and appearances of the objects. Only a small fraction of local features within any given image are associated with a particular caption word, and captions may contain irrelevant words not associated with any image object. We propose a novel algorithm that uses the repetition of feature neighborhoods across training images and a measure of correspondence with caption words to learn meaningful feature configurations (representing named objects). We also introduce a graph-based appearance model that captures some of the structure of an object by encoding the spatial relationships among the local visual features. In an iterative procedure, we use language (the words) to drive a perceptual grouping process that assembles an appearance model for a named object. Results of applying our method to three data sets in a variety of conditions demonstrate that, from complex, cluttered, real-world scenes with noisy captions, we can learn both the names and appearances of objects, resulting in a set of models invariant to translation, scale, orientation, occlusion, and minor changes in viewpoint or articulation. These named models, in turn, are used to automatically annotate new, uncaptioned images, thereby facilitating keyword-based image retrieval.
Stichworte
Language-vision integration;
perceptual grouping;
appearance models;
object recognition;
image annotation
Erscheinungsjahr
2010
Zeitschriftentitel
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
Band
32
Ausgabe
1
Seite(n)
148-164
ISSN
0162-8828
Page URI
https://pub.uni-bielefeld.de/record/1589720
Zitieren
Jamieson M, Fazly A, Stevenson S, Dickinson S, Wachsmuth S. Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. 2010;32(1):148-164.
Jamieson, M., Fazly, A., Stevenson, S., Dickinson, S., & Wachsmuth, S. (2010). Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 32(1), 148-164. https://doi.org/10.1109/TPAMI.2008.283
Jamieson, Michael, Fazly, Afsaneh, Stevenson, Suzanne, Dickinson, Sven, and Wachsmuth, Sven. 2010. “Using Language to Learn Structured Appearance Models for Image Annotation”. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 32 (1): 148-164.
Jamieson, M., Fazly, A., Stevenson, S., Dickinson, S., and Wachsmuth, S. (2010). Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 32, 148-164.
Jamieson, M., et al., 2010. Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 32(1), p 148-164.
M. Jamieson, et al., “Using Language to Learn Structured Appearance Models for Image Annotation”, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 32, 2010, pp. 148-164.
Jamieson, M., Fazly, A., Stevenson, S., Dickinson, S., Wachsmuth, S.: Using Language to Learn Structured Appearance Models for Image Annotation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. 32, 148-164 (2010).
Jamieson, Michael, Fazly, Afsaneh, Stevenson, Suzanne, Dickinson, Sven, and Wachsmuth, Sven. “Using Language to Learn Structured Appearance Models for Image Annotation”. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 32.1 (2010): 148-164.
Daten bereitgestellt von European Bioinformatics Institute (EBI)
Zitationen in Europe PMC
Daten bereitgestellt von Europe PubMed Central.
25 References
Daten bereitgestellt von Europe PubMed Central.
AUTHOR UNKNOWN, 0
Weakly Supervised Learning of Part-Based Spatial Models for Visual Object Recognition
crandall, Proc European Conf Computer Vision (), 2006
crandall, Proc European Conf Computer Vision (), 2006
Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary
duygulu, Proc European Conf Computer Vision 4(), 2002
duygulu, Proc European Conf Computer Vision 4(), 2002
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
PCA-SIFT: A More Distinctive Representation for Local Image Descriptors
ke, Proc IEEE CS Conf Computer Vision and Pattern Recognition (), 2004
ke, Proc IEEE CS Conf Computer Vision and Pattern Recognition (), 2004
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
Supervised learning of semantic classes for image annotation and retrieval.
Carneiro G, Chan AB, Moreno PJ, Vasconcelos N., IEEE Trans Pattern Anal Mach Intell 29(3), 2007
PMID: 17224611
Carneiro G, Chan AB, Moreno PJ, Vasconcelos N., IEEE Trans Pattern Anal Mach Intell 29(3), 2007
PMID: 17224611
A Statistical Model for General Contextual Object Recognition
carbonetto, Proc European Conf Computer Vision (), 2004
carbonetto, Proc European Conf Computer Vision (), 2004
AUTHOR UNKNOWN, 0
Flexible spatial configuration of local image features.
Carneiro G, Jepson AD., IEEE Trans Pattern Anal Mach Intell 29(12), 2007
PMID: 17934220
Carneiro G, Jepson AD., IEEE Trans Pattern Anal Mach Intell 29(12), 2007
PMID: 17934220
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
Modeling semantic aspects for cross-media image indexing.
Monay F, Gatica-Perez D., IEEE Trans Pattern Anal Mach Intell 29(10), 2007
PMID: 17699924
Monay F, Gatica-Perez D., IEEE Trans Pattern Anal Mach Intell 29(10), 2007
PMID: 17699924
AUTHOR UNKNOWN, 0
AUTHOR UNKNOWN, 0
Video Data Mining Using Configurations of Viewpoint Invariant Regions
sivic, Proc IEEE CS Conf Computer Vision and Pattern Recognition (), 2004
sivic, Proc IEEE CS Conf Computer Vision and Pattern Recognition (), 2004
Export
Markieren/ Markierung löschen
Markierte Publikationen
Web of Science
Dieser Datensatz im Web of Science®Quellen
PMID: 19926905
PubMed | Europe PMC
Suchen in