Bayesian reasoning on qualitative descriptions from images and speech

Image understanding denotes not only the ability to extract specific, non-numerical information from images, but it implies also reasoning about the extracted information. We propose a qualitative representation for image understanding results, which is suitable for reasoning with Bayesian networks. Our qualitative representation is enhanced with probabilistic information to represent uncertainties and errors in the understanding of noisy sensory data. The probabilistic information is supplied to a Bayesian network in order to find the most plausible interpretation. We apply this approach for the integration of image and speech understanding in a scenario where we want to find objects in a visually observed scene which are verbally described by a human. Results demonstrate the performance of our approach. (C) 2000 Elsevier Science B.V. All rights reserved.