PUB - Publikationen an der Universität Bielefeld

A great part of human behavior is based on visual cognition, the processing of visual information about external objects. For goal-directed behavior, two functions of visual cognition seem especially important. The first one is object recognition. Objects in the environment must be identified as belonging to an object category, so that they can be used to accomplish a given task. The second function is short-term recognition. It must be recognized whether an object in the environment has been viewed recently, so that current behavior involving the object can be related to previous behavior. Both functions share a common constraint: They must be fulfilled across distinct episodes of visual processing, which are interrupted by changes in processing demands.

For object recognition, visual processing episodes lead to a problem of selective integration. That is, it must be decided whether object information from the current episode should update and thus be integrated with object representations from the previous episode. Alternatively, object representations from two successive episodes are retained separately. This decision is critical. Updating and integration should enable a cumulative and fast object recognition. However, integration should also conceal object changes across episodes by leaving no separate representations that can be compared. Separation should improve change perception but impair object recognition, because limited visual processing resources for object recognition must be split between the separate representations.

For short-term recognition visual processing episodes lead to a problem of matching. That is, an object from the current episode must be matched against object representations, not only from the previous but from several episodes in the recent past (irrespective of whether objects are categorized).

The overarching goal of the present dissertation is to make a first step in understanding how the mechanisms underlying object recognition and short-term recognition operate across visual processing episodes, and how they solve the two problems. In five empirical studies, we investigated key issues that must be addressed before a theoretical account of object and short-term recognition across visual processing episodes can be given.

The first three studies focused on object recognition across visual processing episodes of eye fixations. Fixations are periods of visual information uptake, in which the eyes stand relatively still. They are separated by rapid saccadic eye movements. Saccades are necessary for object recognition, because they direct the central fovea of the eye's retina at interesting objects, allowing high-acuity inspection. However, saccades also disrupt visual input and displace and alter the retinal images of objects. Therefore, saccades dissect visual information processing into distinct episodes of fixations, which the mechanisms for object recognition must accommodate. In two studies (Poth, Herwig, & Schneider, 2015; Poth & Schneider, 2016a), we investigated how the selective integration problem is solved to support object recognition across successive fixations. We assessed the recent hypothesis (Schneider, 2013) that the problem is solved by a mechanism testing for correspondence (``object continuity'') between an object before and after a saccade. If object correspondence is established, the object before and after the saccade should be integrated into a common representation. In contrast, if object correspondence is broken, the object before and after the saccade should be represented separately. Separation should allow to compare the two representations, improving the discrimination of transsaccadic object displacements. At the same time, however, object recognition of the object after the saccade should be impaired, because the necessary visual processing resources had to be split between the two representations. Results were consistent with this hypothesis. Breaking object correspondence by briefly blanking an object after a saccade to it improved the discrimination of displacements of the object but impaired object recognition. Thus, the object correspondence mechanism seems to impact on object recognition after the saccade. Further experiments investigated the nature of object correspondence. They showed that object recognition was impaired when object correspondence was broken by changing an object's contrast-polarity (and luminance), its color-and-luminance, and its color alone. Together with the initial finding, this indicates that object correspondence is based on spatiotemporal as well as on the surface features of objects. In the third study (Poth & Schneider, 2016b, submitted), we went on to test the limits of object recognition across saccades. Because object recognition relies on limited visual processing resources, it can only be achieved for a few objects at a time. Here, we examined if different objects must compete for these resources across saccades. If this was the case, visual processing after a saccade would be slowed down as more and more objects are viewed before the saccade. Our findings show that this is the case, but only if the objects are task-relevant. Therefore, the findings support a key prediction of a recent theory, namely that the importance of an object representation determines whether it will survive a saccade and take up limited processing resources afterwards (Schneider, 2013).

With the fourth study (Poth & Schneider, 2016c), we turned from the processing episodes of successive eye fixations to those defined by appearing and disappearing objects and associated task-requirements. We asked about the relationship between the mechanisms underlying object recognition and those underlying short-term recognition. Visual processing for object recognition is assumed to be complete when an object has entered a limited-capacity visual working memory, where the object becomes available for being reported. We investigated if encoding into visual working memory is not only required for object recognition in the current episode, but also for short-term recognition in upcoming episodes. Supporting this notion, we found that objects that supposedly had not reached visual working memory were not available for later short-term recognition. This finding argues that the initial steps of visual processing before encoding into visual working memory are not sufficient for short-term recognition in later episodes. Therefore, visual working memory may contribute to the solution of the matching problem by limiting the amount of information considered in a short-term recognition task.

Finally, in the fifth study (Poth & Schneider, 2016d, submitted), we investigated short-term recognition further, asking how short-term recognition in a later processing episode can be prepared in advance. We assessed how prioritizing among objects represented in visual working memory impacts on two distinct components of performance in an upcoming short-term recognition task. Our results showed that such a prioritization improves memory-retention in visual working memory but also accelerates visual processing of objects for short-term recognition in a future episode. This indicates that changes in processing priorities contribute to ongoing solutions of the matching problem of short-term recognition.

Taken together, the five studies show how mechanisms of object and short-term recognition address specific problems arising from the dissection of visual processing into distinct episodes. As such, the studies implicate visual processing episodes as a source of problems for object and short-term recognition, which is neglected in most contemporary research. Conversely, however, the studies also invite speculation about the functional value of visual processing episodes for visual cognition.