Learning to integrate arbitrary signals from vision and touch

Ernst MO (2007)
Journal of Vision 7(5).

Journal Article | Published | English

No fulltext has been uploaded

Abstract
When different perceptual signals of the same physical property are integrated, for example, an objects' size, which can be seen and felt, they form a more reliable sensory estimate ( e. g., M. O. Ernst & M. S. Banks, 2002). This, however, implies that the sensory system already knows which signals belong together and how they relate. In other words, the system has to know the mapping between the signals. In a Bayesian model of cue integration, this prior knowledge can be made explicit. Here, we ask whether such a mapping between two arbitrary sensory signals from vision and touch can be learned from their statistical co- occurrence such that they become integrated. In the Bayesian framework, this means changing the belief about the distribution of the stimuli. To this end, we trained subjects with stimuli that are usually unrelated in the world-the luminance of an object ( visual signal) and its stiffness ( haptic signal). In the training phase, we then presented subjects with combinations of these two signals, which were artificially correlated, and thus, we introduced a new mapping between them. For example, the stiffer the object, the brighter it was. We measured the influence of learning by comparing discrimination performance before and after training. The prediction is that integration makes discrimination worse for stimuli, which are incongruent with the newly learned mapping, because integration would cause this incongruency to disappear perceptually. The more certain subjects are about the new mapping, the stronger should the influence be on discrimination performance. Thus, learning in this context is about acquiring beliefs. We found a significant change in discrimination performance before and after training when comparing trials with congruent and incongruent stimuli. After training, discrimination thresholds for the incongruent stimuli are increased relative to thresholds for congruent stimuli, suggesting that subjects learned effectively to integrate the two formerly unrelated signals.
Publishing Year
ISSN
PUB-ID

Cite this

Ernst MO. Learning to integrate arbitrary signals from vision and touch. Journal of Vision. 2007;7(5).
Ernst, M. O. (2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7(5).
Ernst, M. O. (2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision 7.
Ernst, M.O., 2007. Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7(5).
M.O. Ernst, “Learning to integrate arbitrary signals from vision and touch”, Journal of Vision, vol. 7, 2007.
Ernst, M.O.: Learning to integrate arbitrary signals from vision and touch. Journal of Vision. 7, (2007).
Ernst, Marc O. “Learning to integrate arbitrary signals from vision and touch”. Journal of Vision 7.5 (2007).
This data publication is cited in the following publications:
This publication cites the following data publications:

37 Citations in Europe PMC

Data provided by Europe PubMed Central.

Touch influences perceived gloss.
Adams WJ, Kerrigan IS, Graf EW., Sci Rep 6(), 2016
PMID: 26915492
Spatiotemporal Processing in Crossmodal Interactions for Perception of the External World: A Review.
Hidaka S, Teramoto W, Sugita Y., Front Integr Neurosci 9(), 2015
PMID: 26733827
Musicians are more consistent: Gestural cross-modal mappings of pitch, loudness and tempo in real-time.
Kussner MB, Tidhar D, Prior HM, Leech-Wilkinson D., Front Psychol 5(), 2014
PMID: 25120506
How learning to abstract shapes neural sound representations.
Ley A, Vroomen J, Formisano E., Front Neurosci 8(), 2014
PMID: 24917783
Verbal and novel multisensory associative learning in adults.
Fifer JM, Barutchu A, Shivdasani MN, Crewther SG., F1000Res 2(), 2013
PMID: 24627770
Learning not to feel: reshaping the resolution of tactile perception.
Omrani M, Lak A, Diamond ME., Front Syst Neurosci 7(), 2013
PMID: 23847478
Constructive perception of self-motion.
Holly JE, McCollum G., J Vestib Res 18(5-6), 2008
PMID: 19542599

Export

0 Marked Publications

Open Data PUB

Web of Science

View record in Web of Science®

Sources

PMID: 18217847
PubMed | Europe PMC

Search this title in

Google Scholar