Dr. Marcin Leszczyński from the Jagiellonian University Cognitive Science Department and the Department of Neurological Surgery, Columbia University Medical Center in New York talks to PAP about the ways we perceive images and sounds, why individual regions of our brain - the so-called the auditory and visual brain - complement each other and give our mind full information about the world around us. And why 'mind wandering', our mental 'highs', favours creativity.

PAP: Why do you put electrodes into people's brains?

Dr. Marcin Leszczyński: I do not put them in myself, a neurosurgeon does that, but I actually do participate in such procedures. I work with neurosurgeons and neurologists in several hospitals in the United States and Germany where patients with drug-resistant epilepsy are treated. One treatment option if drug therapy doesn't work is to remove the part of the brain where the epileptic seizures originate. To remove such a fragment of the brain, it must first be precisely located. This is one of the reasons why electrodes are inserted inside the brain.

You also need to make sure that it is not located in a place that is responsible for any important processes. By removing it, we could do more harm than good - for example, damage the patient's ability to form new memories or the ability to speak.

After the electrode implantation surgery, the patient's brain function is monitored 24 hours a day, for several days. I use this time to conduct measurements that will allow me to understand how the brain functions while performing various cognitive activities - for example, watching a movie, listening to a podcast, concentrating or memorising information. Since we usually implant several dozen, and sometimes even several hundred electrodes, we obtain a huge amount of data, which we then subject to thorough analyses in my laboratory.

PAP: I know that your research has shown that we do not hear and see at the same time, but alternately: image - sound. How is this possible?

M.L.: When you watch a movie, look at a picture or observe your surroundings, instead of staring fixedly at one point, you move your eyes, shifting them from one point of the visual scene to another. You absorb visual information actively by focusing on a part of the visual scene when you stare at one point ('fixation') and then quickly moving your eyes ('saccade') to another spot. This happens because humans, as well as other species of primates, have special cells in the retina of the eye that convert the light reaching the eye into an electrical impulse. Because these cells are particularly densely packed in one place (the macula in the retina), saccades allow this part of the retina to be directed to subsequent elements of the environment and subject them to more precise processing.

Research shows that - contrary to our impression - we see clearly only a small fragment of what is in front of our eyes. It is similar with colour - we see colours only in a small area at which the macula is directed. We register everything else blurrily and colourlessly. The brain fills in the missing information and creates the impression of a rich and complex experience.

It may be surprising that the area of this sharp vision is about the size of a fingernail. Saccadic movements allow us to direct this point of sharp and clear vision to various elements of the surrounding reality and thus scan the environment. During subsequent fixations, information about what surrounds us is sent to the brain. We can imagine it as a series of frames, like in a movie. Even though the brain receives individual samples from each fixation point, our experience is different. We do not see consecutive frames, but rather have a sense of a stable and continuous visual experience. In my lab, we study the physiological processes that allow the brain to 'put it together'.

PAP: How does hearing fit into this?

M.L.: During saccadic movements, our ability to register visual information is limited. We then see very little. Because we make saccades all the time (on average several times per second), our brains have to somehow cope with the missing information. The results of my research show that the brain copes by using information from other senses, especially the sense of hearing. During saccadic movements, when we see very little, the auditory brain provides the missing information about what is happening around us. My research also suggests that the interaction between the auditory and visual brain provides a sense of continuity, even though the information the brain receives is just a sequence of frames.

PAP: What happens when I close my eyes?

M.L.: It is similar - the auditory brain is more excitable, which means it is easier for it to register auditory information. These are two very similar situations. Even though we do not close our eyes during saccadic movements, visual information is actively suppressed by the brain. Both during saccadic movements and when we close our eyes, the auditory brain takes over as the dominant source of information.

PAP: This would explain why people who cannot see usually have such good hearing.

M.L.: Indeed, research confirms that blind people have better hearing than sighted people. It is worth adding, however, that this mainly applies to people who have been blind since birth or from the earliest years of life. Additionally, different features of auditory information are improved to varying degrees.

PAP: Does this knowledge have any practical implications?

M.L.: In my laboratory, thanks to funding from the Polish National Science Centre, we primarily conduct basic research. We try to understand how it is that we can focus on something or remember information. What changes in brain function allow us to remember information? What determines the way of exploring the environment, i.e. the sequence of saccades and fixations? And how does the brain work in different situations?

The answer to these questions has a number of practical consequences. For example, the saccade and fixation sequences I mentioned earlier are used in the diagnosis of children with autism spectrum disorders. Many mental and neurological disorders are characterized by very specific changes in the exploration of the environment. This makes saccade parameters a very useful diagnostic tool. It is similar with the way the brain works. For example, research shows that the brain, especially the temporal lobes, functions differently in people at increased risk of Alzheimer's disease many years or even decades before the first symptoms appear. This makes the way the brain works useful in diagnostic processes. Of course, these are just two examples, and there are many more.

In my laboratory, we also conduct research on the phenomenon of mind wandering. Let me go back to the example of watching a movie: even if you are interested in what you are watching, you rarely manage to stay focused all the time. In most cases, you are able to focus your attention for a moment. You actually listen and actively process the information that reaches you. And then you drift away from the here and now. You may still be looking at the screen, but you process the information you receive to a much lesser extent. Of course, how much you are able to focus and how much your thoughts wander depends on many factors.

It may seem surprising, but studies indicate that for about 40 percent of the time we are not focused on what we are currently doing. This is a huge amount of time when our attention appears to work less effectively. We investigate what happens when people's thoughts wander. We are trying to understand the reasons and brain mechanisms for this wandering. This, of course, also has a number of practical consequences

PAP: When the pandemic started, students were asked what they were doing while listening to lectures remotely. One of the respondents replied that she was only able to concentrate on these lectures when she was shooting a bow.

M.L.: Yes, it is understandable, although it may seem counterintuitive. Research shows that engaging in additional tasks, such as archery, can improve focus and attention. There is a theory called perceptual load theory. According to it, if you perform a simple cognitive task (e.g. listening to a lecture on a topic that is familiar to you), you do not use all of your attentional resources. The saved resources are automatically directed to processing information irrelevant to the task being performed (e.g. your mind wanders and you thing about what you will do after the lecture). However, if the task is sufficiently difficult and consumes all resources - then, paradoxically, it is easier to focus (because you do not have free resources that could be used to process irrelevant information). Therefore, performing an additional activity such as shooting a bow while listening to a lecture may engage the saved attentional resources and, consequently, support focusing on the lecture and improve the amount of information remembered from such a lecture.

PAP: It would seem that we need these 'highs' for something, if they were not eliminated in the evolutionary process as too dangerous for the survival of the species.

M.L.: Our minds wander very often. As I mentioned, research shows that for about 40 percent of the time we are not focused on what we are currently doing. As a result, it is more difficult for us to register and react to changes in the environment, which can sometimes lead to serious consequences (e.g. when our thoughts wander while driving a car). The prevailing belief in psychology and neuroscience is that mind wandering is a functionless limitation of cognitive abilities. However, the frequency of this phenomenon and, above all, the observation that similar processes occur in different species - in humans, monkeys and rodents - lead me to believe that, apart from the obvious impairment of attention, they play an important role. What exactly is this role? We intend to answer this question in the course of our current research. My previous research, conducted in cooperation with researchers from the University Hospital in Bonn, shows that mind wandering supports creative thinking.

PAP: Does this mean that we shouldn't blame ourselves for the fact that our thoughts wander too often?

M.L.: I think that mind wandering is a necessary process, although of course it is worth exercising moderation. One possibility that we are exploring in my lab is that mind wandering creates the perfect conditions for memory trace consolidation. This process is necessary for our experiences to take on a more lasting form and turn into memories.

PAP: I was taught that memory strengthens during sleep.

M.L.: That is true. Consolidation processes also occur during sleep. Mind wandering, however, creates conditions similar to those that occur during sleep and that is why it may be an ideal moment, an introduction to consolidation processes. (PAP)

Interview by Mira Suchodolska

PAP - Science in Poland

mir/ ral/ kap/

tr. RL