Multimodal interaction in an augmented reality scenario

Heidemann G, Bax I, Bekel H, Bauckhage C, Wachsmuth S, Fink GA, Pinz A, Ritter H, Sagerer G (2004)
In: Proceedings of the 6th international conference on Multimodal interfaces. New York, USA: ACM: 53-60.

Conference Paper | Published | English

No fulltext has been uploaded

Author
; ; ; ; ; ; ; ;
Abstract
We describe an augmented reality system designed for online acquisition of visual knowledge and retrieval of memorized objects. The system relies on a head mounted camera and display, which allow the user to view the environment together with overlaid augmentations by the system. In this setup, communication by hand gestures and speech is mandatory as common input devices like mouse and keyboard are not available. Using gesture and speech, basically three types of tasks must be handled: (i) Communication with the system about the environment, in particular, directing attention towards objects and commanding the memorization of sample views; (ii) control of system operation, e.g. switching between display modes; and (iii) re-adaptation of the interface itself in case communication becomes unreliable due to changes in external factors, such as illumination conditions. We present an architecture to manage these tasks and describe and evaluate several of its key elements, including modules for pointing gesture recognition, menu control based on gesture and speech, and control strategies to cope with situations when vision becomes unreliable and has to be re-adapted by speech.
Publishing Year
Conference
ICMI '04 Sixth International Conference on Multimodal Interfaces
Location
New York, USA
Conference Date
2004-10-13 – 2004-10-15
PUB-ID

Cite this

Heidemann G, Bax I, Bekel H, et al. Multimodal interaction in an augmented reality scenario. In: Proceedings of the 6th international conference on Multimodal interfaces. New York, USA: ACM; 2004: 53-60.
Heidemann, G., Bax, I., Bekel, H., Bauckhage, C., Wachsmuth, S., Fink, G. A., Pinz, A., et al. (2004). Multimodal interaction in an augmented reality scenario. Proceedings of the 6th international conference on Multimodal interfaces, 53-60.
Heidemann, G., Bax, I., Bekel, H., Bauckhage, C., Wachsmuth, S., Fink, G. A., Pinz, A., Ritter, H., and Sagerer, G. (2004). “Multimodal interaction in an augmented reality scenario” in Proceedings of the 6th international conference on Multimodal interfaces (New York, USA: ACM), 53-60.
Heidemann, G., et al., 2004. Multimodal interaction in an augmented reality scenario. In Proceedings of the 6th international conference on Multimodal interfaces. New York, USA: ACM, pp. 53-60.
G. Heidemann, et al., “Multimodal interaction in an augmented reality scenario”, Proceedings of the 6th international conference on Multimodal interfaces, New York, USA: ACM, 2004, pp.53-60.
Heidemann, G., Bax, I., Bekel, H., Bauckhage, C., Wachsmuth, S., Fink, G.A., Pinz, A., Ritter, H., Sagerer, G.: Multimodal interaction in an augmented reality scenario. Proceedings of the 6th international conference on Multimodal interfaces. p. 53-60. ACM, New York, USA (2004).
Heidemann, Gunther, Bax, Ingo, Bekel, Holger, Bauckhage, Christian, Wachsmuth, Sven, Fink, Gernot A., Pinz, A., Ritter, Helge, and Sagerer, Gerhard. “Multimodal interaction in an augmented reality scenario”. Proceedings of the 6th international conference on Multimodal interfaces. New York, USA: ACM, 2004. 53-60.
This data publication is cited in the following publications:
This publication cites the following data publications:

Export

0 Marked Publications

Open Data PUB

Search this title in

Google Scholar
ISBN Search