Attention and Memory in Overt Visual Search

Visual search is a behaviour that occurs many times every day. When we search for our keys before leaving the house or when we look for a familiar face in a crowd, we perform a visual search, that is, the search for a target object among a set of more or less similar other objects or distractors. In a complex visual environment, we are typically not able to spot the target immediately, and have to inspect and classify individual objects one by one a process known as serial visual search. In this project, we will use the measurement of eye movements together with either electroencephalography (EEG) or functional magnetic resonance imaging (fMRI) to investigate how targets and distractors are processed during serial visual search when participants are allowed to move their eyes freely (overt visual search). Eye movements are most often avoided in search tasks because they can have adverse effects on EEG and fMRI data. We will show that the detrimental effects of eye movements on EEG and MRI can be overcome and that the co-registration of these data can yield important insights into the moment-by-moment processing of targets and distractors. Visual search has always been one of the key paradigms to investigate attention. Over the last 15 years the possible involvement of memory in visual search has also become an important and highly debated research topic. The overall objective of this project is to gain a better understanding of the processes involved in overt visual search on an electrophysiological and neural level, in particular with respect to attention and memory, two key areas of cognitive neuroscience. We propose two streams of research corresponding to these areas. In the first stream (attention) we will try to explore bottom-up and top-down influences on the P3, an electrophysiological marker of target processing. In the second stream (memory) we investigate electrophysiological and neural indicators of memory in serial visual search. Our research in this area will produce significant insights into the moment-by-moment deployment of attention and memory in overt serial visual search.

Visual search is a behaviour that humans and higher organisms perform multiple times each day. When we leave the house and search for our car keys and an umbrella, or when we go shopping in the supermarket we perform visual search. Investigating visual search is important because it provides insight into the fundamental process of attention. In a visual search, attention moves from one object or location to the next until the target object is fount (or not). During search, we must also remember previously visited locations or objects to avoid searching the same place multiple times. Visual search has been investigated for several decades. There is psychological research that tracks the movements of the eyes. This is useful because a person's attention is most likely where their gaze is. Neuroscientific research uses the recording of electrical activity from the scalp (electroencephalography EEG) or measures blood oxygenation in the brain (functional magnet resonance imaging fMRI). This allows to investigate brain activity during visual search. For mostly technical reasons eye tracking has rarely been used in combination with EEG or fMRI. Employing cutting-edge research methodology we succeeded in identifying neural correlates of visual search. Participants in our studies had to search for target letters in displays with other letters (distractors) presented on a computer screen. The participants' eye movements were recorded in order to determine which object was being fixated at any given moment. Simultaneously, we recorded either EEG or fMRI. Using advanced analysis methods it was possible, for example, to determine what happened in the brain of the participants when they looked at a target letter in comparison to a distractor. We were able to determine that brain areas responsible for goal directed attention were involved in finding a target, as well as how targets that have been found are stored in the brain. We also found that a brain region, the temporo-parietal junction, distinguished between task relevant and irrelevant information during visual search. Our findings help to better understand attention as a basic function of the healthy human brain. They also provide a basis to understand problems caused by brain lesions in tasks that require attention.