Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps

Yan R, Rodemann T, Wrede B (2013)
IEEE Transactions on Autonomous Mental Development 5(4): 273-287.

Journal Article | Published | English

No fulltext has been uploaded

Author
; ;
Abstract
For sound localization, the binaural auditory system of a robot needs audio-motor maps, which represent the relationship between certain audio features and the position of the sound source. This mapping is normally learned during an offline calibration in controlled environments, but we show that using computational audiovisual scene analysis (CAVSA), it can be adapted online in free interaction with a number of a priori unknown speakers. CAVSA enables a robot to understand dynamic dialog scenarios, such as the number and position of speakers, as well as who is the current speaker. Our system does not require specific robot motions and thus can work during other tasks. The performance of online-adapted maps is continuously monitored by computing the difference between online-adapted and offline-calibrated maps and also comparing sound localization results with ground truth data (if available). We show that our approach is more robust in multiperson scenarios than the state of the art in terms of learning progress. We also show that our system is able to bootstrap with a randomized audio-motor map and adapt to hardware modifications that induce a change in audio-motor maps.
Publishing Year
ISSN
eISSN
PUB-ID

Cite this

Yan R, Rodemann T, Wrede B. Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps. IEEE Transactions on Autonomous Mental Development. 2013;5(4):273-287.
Yan, R., Rodemann, T., & Wrede, B. (2013). Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps. IEEE Transactions on Autonomous Mental Development, 5(4), 273-287.
Yan, R., Rodemann, T., and Wrede, B. (2013). Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps. IEEE Transactions on Autonomous Mental Development 5, 273-287.
Yan, R., Rodemann, T., & Wrede, B., 2013. Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps. IEEE Transactions on Autonomous Mental Development, 5(4), p 273-287.
R. Yan, T. Rodemann, and B. Wrede, “Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps”, IEEE Transactions on Autonomous Mental Development, vol. 5, 2013, pp. 273-287.
Yan, R., Rodemann, T., Wrede, B.: Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps. IEEE Transactions on Autonomous Mental Development. 5, 273-287 (2013).
Yan, Rujiao, Rodemann, Tobias, and Wrede, Britta. “Computational Audiovisual Scene Analysis in Online Adaptation of Audio-Motor Maps”. IEEE Transactions on Autonomous Mental Development 5.4 (2013): 273-287.
This data publication is cited in the following publications:
This publication cites the following data publications:

Export

0 Marked Publications

Open Data PUB

Web of Science

View record in Web of Science®

Search this title in

Google Scholar