Learning to integrate arbitrary signals from vision and touch

Ernst MO (2007)
Journal of Vision 7(5): 7.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
Abstract / Bemerkung
When different perceptual signals of the same physical property are integrated, for example, an objects' size, which can be seen and felt, they form a more reliable sensory estimate ( e. g., M. O. Ernst & M. S. Banks, 2002). This, however, implies that the sensory system already knows which signals belong together and how they relate. In other words, the system has to know the mapping between the signals. In a Bayesian model of cue integration, this prior knowledge can be made explicit. Here, we ask whether such a mapping between two arbitrary sensory signals from vision and touch can be learned from their statistical co- occurrence such that they become integrated. In the Bayesian framework, this means changing the belief about the distribution of the stimuli. To this end, we trained subjects with stimuli that are usually unrelated in the world-the luminance of an object ( visual signal) and its stiffness ( haptic signal). In the training phase, we then presented subjects with combinations of these two signals, which were artificially correlated, and thus, we introduced a new mapping between them. For example, the stiffer the object, the brighter it was. We measured the influence of learning by comparing discrimination performance before and after training. The prediction is that integration makes discrimination worse for stimuli, which are incongruent with the newly learned mapping, because integration would cause this incongruency to disappear perceptually. The more certain subjects are about the new mapping, the stronger should the influence be on discrimination performance. Thus, learning in this context is about acquiring beliefs. We found a significant change in discrimination performance before and after training when comparing trials with congruent and incongruent stimuli. After training, discrimination thresholds for the incongruent stimuli are increased relative to thresholds for congruent stimuli, suggesting that subjects learned effectively to integrate the two formerly unrelated signals.
Bayesian models of perception; touch; vision; statistical learning; cue integration
Journal of Vision
Page URI


Ernst MO. Learning to integrate arbitrary signals from vision and touch. Journal of Vision. 2007;7(5):7.
Ernst, M. O. (2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7(5), 7. https://doi.org/10.1167/7.5.7
Ernst, Marc O. 2007. “Learning to integrate arbitrary signals from vision and touch”. Journal of Vision 7 (5): 7.
Ernst, M. O. (2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision 7, 7.
Ernst, M.O., 2007. Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7(5), p 7.
M.O. Ernst, “Learning to integrate arbitrary signals from vision and touch”, Journal of Vision, vol. 7, 2007, pp. 7.
Ernst, M.O.: Learning to integrate arbitrary signals from vision and touch. Journal of Vision. 7, 7 (2007).
Ernst, Marc O. “Learning to integrate arbitrary signals from vision and touch”. Journal of Vision 7.5 (2007): 7.

65 Zitationen in Europe PMC

Daten bereitgestellt von Europe PubMed Central.

Suprathreshold Motion Perception in Anisometropic Amblyopia: Interocular Speed Matching and the Pulfrich Effect.
Maehara G, Araki S, Yoneda T, Thompson B, Miki A., Optom Vis Sci 96(6), 2019
PMID: 31107841
The Role of Intrinsic and Extrinsic Sensory Factors in Sweetness Perception of Food and Beverages: A Review.
Wang QJ, Mielby LA, Junge JY, Bertelsen AS, Kidmose U, Spence C, Byrne DV., Foods 8(6), 2019
PMID: 31208021
Semantic Associations Dominate Over Perceptual Associations in Vowel-Size Iconicity.
Hoshi H, Kwon N, Akita K, Auracher J., Iperception 10(4), 2019
PMID: 31321019
Active Haptic Perception in Robots: A Review.
Seminara L, Gastaldo P, Watt SJ, Valyear KF, Zuher F, Mastrogiovanni F., Front Neurorobot 13(), 2019
PMID: 31379549
When preschoolers follow their eyes and older children follow their noses: visuo-olfactory social affective matching in childhood.
Cavazzana A, Wesarg C, Parish-Morris J, Lundström JN, Parma V., Dev Sci 21(1), 2018
PMID: 27859959
Energy exchanges at contact events guide sensorimotor integration.
Farshchian A, Sciutti A, Pressman A, Nisky I, Mussa-Ivaldi FA., Elife 7(), 2018
PMID: 29809144
Five mechanisms of sound symbolic association.
Sidhu DM, Pexman PM., Psychon Bull Rev 25(5), 2018
PMID: 28840520
Peri-personal space as a prior in coupling visual and proprioceptive signals.
Noel JP, Samad M, Doxon A, Clark J, Keller S, Di Luca M., Sci Rep 8(1), 2018
PMID: 30361477
Perceptual attraction in tool use: evidence for a reliability-based weighting mechanism.
Debats NB, Ernst MO, Heuer H., J Neurophysiol 117(4), 2017
PMID: 28100656
Optimal visual-haptic integration with articulated tools.
Takahashi C, Watt SJ., Exp Brain Res 235(5), 2017
PMID: 28214998
A simple and efficient method to enhance audiovisual binding tendencies.
Odegaard B, Wozny DR, Shams L., PeerJ 5(), 2017
PMID: 28462016
Trust in haptic assistance: weighting visual and haptic cues based on error history.
Gibo TL, Mugge W, Abbink DA., Exp Brain Res 235(8), 2017
PMID: 28534068
Use of cues in virtual reality depends on visual feedback.
Fulvio JM, Rokers B., Sci Rep 7(1), 2017
PMID: 29167491
The diversity rank-score function for combining human visual perception systems.
Schweikert C, Mulia D, Sanchez K, Hsu DF., Brain Inform 3(1), 2016
PMID: 27747600
Touch influences perceived gloss.
Adams WJ, Kerrigan IS, Graf EW., Sci Rep 6(), 2016
PMID: 26915492
Perceptual learning shapes multisensory causal inference via two distinct mechanisms.
McGovern DP, Roudaia E, Newell FN, Roach NW., Sci Rep 6(), 2016
PMID: 27091411
Bayesian Alternation during Tactile Augmentation.
Goeke CM, Planera S, Finger H, König P., Front Behav Neurosci 10(), 2016
PMID: 27774057
On the combination of two visual cognition systems using combinatorial fusion.
Batallones A, Sanchez K, Mott B, Coffran C, Frank Hsu D., Brain Inform 2(1), 2015
PMID: 27747501
Spatiotemporal Processing in Crossmodal Interactions for Perception of the External World: A Review.
Hidaka S, Teramoto W, Sugita Y., Front Integr Neurosci 9(), 2015
PMID: 26733827
Is accurate mapping of EMG signals on kinematics needed for precise online myoelectric control?
Jiang N, Vujaklija I, Rehbaum H, Graimann B, Farina D., IEEE Trans Neural Syst Rehabil Eng 22(3), 2014
PMID: 24235278
The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges.
Farina D, Jiang N, Rehbaum H, Holobar A, Graimann B, Dietl H, Aszmann OC., IEEE Trans Neural Syst Rehabil Eng 22(4), 2014
PMID: 24760934
Compressive mapping of number to space reflects dynamic encoding mechanisms, not static logarithmic transform.
Cicchini GM, Anobile G, Burr DC., Proc Natl Acad Sci U S A 111(21), 2014
PMID: 24821771
How learning to abstract shapes neural sound representations.
Ley A, Vroomen J, Formisano E., Front Neurosci 8(), 2014
PMID: 24917783
Musicians are more consistent: Gestural cross-modal mappings of pitch, loudness and tempo in real-time.
Küssner MB, Tidhar D, Prior HM, Leech-Wilkinson D., Front Psychol 5(), 2014
PMID: 25120506
Cross-cultural differences in crossmodal correspondences between basic tastes and visual features.
Wan X, Woods AT, van den Bosch JJ, McKenzie KJ, Velasco C, Spence C., Front Psychol 5(), 2014
PMID: 25538643
Smelling shapes: crossmodal correspondences between odors and shapes.
Hanson-Vaux G, Crisinel AS, Spence C., Chem Senses 38(2), 2013
PMID: 23118203
Sensorimotor priors in nonstationary environments.
Narain D, van Beers RJ, Smeets JB, Brenner E., J Neurophysiol 109(5), 2013
PMID: 23235999
Verbal and novel multisensory associative learning in adults.
Fifer JM, Barutchu A, Shivdasani MN, Crewther SG., F1000Res 2(), 2013
PMID: 24627770
Perceptual integration for qualitatively different 3-D cues in the human brain.
Dövencioğlu D, Ban H, Schofield AJ, Welchman AE., J Cogn Neurosci 25(9), 2013
PMID: 23647559
Learning not to feel: reshaping the resolution of tactile perception.
Omrani M, Lak A, Diamond ME., Front Syst Neurosci 7(), 2013
PMID: 23847478
Visual and haptic integration in the estimation of softness of deformable objects.
Cellini C, Kaim L, Drewing K., Iperception 4(8), 2013
PMID: 25165510
Crossmodal correspondences: Innate or learned?
Spence C, Deroy O., Iperception 3(5), 2012
PMID: 23145286
The cognitive neuroscience of crossmodal correspondences.
Spence C, Parise CV., Iperception 3(7), 2012
PMID: 23145291
Combining symbolic cues with sensory input and prior experience in an iterative bayesian framework.
Petzschner FH, Maier P, Glasauer S., Front Integr Neurosci 6(), 2012
PMID: 22905024
Learning from vision-to-touch is different than learning from touch-to-vision.
Wismeijer DA, Gegenfurtner KR, Drewing K., Front Integr Neurosci 6(), 2012
PMID: 23181012
Precision and reliability in animal navigation.
Pfuhl G, Tjelmeland H, Biegler R., Bull Math Biol 73(5), 2011
PMID: 20496009
Influences of multisensory experience on subsequent unisensory processing.
Shams L, Wozny DR, Kim R, Seitz A., Front Psychol 2(), 2011
PMID: 22028697
Computational characterization of visually induced auditory spatial adaptation.
Wozny DR, Shams L., Front Integr Neurosci 5(), 2011
PMID: 22069383
Learning to use an invisible visual signal for perception.
Di Luca M, Ernst MO, Backus BT., Curr Biol 20(20), 2010
PMID: 20933421
The role of visuohaptic experience in visually perceived depth.
Ho YX, Serwe S, Trommershäuser J, Maloney LT, Landy MS., J Neurophysiol 101(6), 2009
PMID: 19357346
Constructive perception of self-motion.
Holly JE, McCollum G., J Vestib Res 18(5-6), 2008
PMID: 19542599
Recalibration of perceived time across sensory modalities.
Hanson JV, Heron J, Whitaker D., Exp Brain Res 185(2), 2008
PMID: 18236035
Benefits of multisensory learning.
Shams L, Seitz AR., Trends Cogn Sci 12(11), 2008
PMID: 18805039

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®

PMID: 18217847
PubMed | Europe PMC

Suchen in

Google Scholar