The researchers reached the conclusion after studying how 20 Deaf people processed a Polish Sign Language version of Edgar Allan Poe's The Fall of the House of Usher. The results were published in Nature Communications.

Although humans have been sharing stories for thousands of years, understanding a narrative requires the brain to perform a complex task. It must combine individual words or signs into sentences, sentences into successive scenes and those scenes into a coherent story.

Scientists have long known that when people listen to speech, different areas of the brain process information at different levels. Some respond to individual words and short fragments of speech, while others follow longer narrative threads.

Until now, however, it was unclear whether a similar mechanism operates when a story is conveyed in sign language through hand movements, facial expressions and the use of space.

An international team of researchers from the University of Warsaw, Jagiellonian University, the Nencki Institute of Experimental Biology and Johns Hopkins University investigated the question.

“Contrary to popular belief, sign languages are fully-fledged natural languages with complex grammar and an extensive lexicon. There are over 300 different sign languages worldwide, and they can differ significantly from each other. Communication in sign languages does not differ fundamentally from communication in spoken languages in terms of their ability to convey content. One of the specific features, however, is the use of space: a signer can arrange individual threads, people they are talking about, objects, or places, and then refer to them during their speech”, says Maria Zimmermann, PhD, from the Faculty of Psychology of the University of Warsaw.

The researchers wanted to determine whether the brain processes such narratives in the same way as speech or whether sign language requires additional mechanisms.

Twenty Deaf people whose first and native language was Polish Sign Language took part in the experiment. During functional magnetic resonance imaging (fMRI), they watched an approximately 20-minute translation of Poe's The Fall of the House of Usher, performed in Polish Sign Language by Deaf actor Adam Stoyanov.

The researchers prepared several modified versions of the recording. In one, the order of individual signs was preserved but the sentence structure was disrupted. In another, entire sentences were rearranged, removing the coherent plot. Other versions contained only short fragments of the material.

This allowed the researchers to determine which areas of the brain responded to individual signs, which were involved in sentence comprehension and which required a longer context to follow the development of the story.

The researchers used a method called interindividual synchronization, which measures the synchronization of activity in different brain areas between individuals.

“This involves measuring the synchronization of activity of different brain areas between individuals. When we listen to a story, watch a film, or read a book, part of what happens in our brain is related to the content being processed. At the same time, many other thoughts, associations, and references emerge, which are specific to each individual. By analysing the degree to which the signal in a given brain region is synchronized between individuals, we can determine the extent to which its activity is universal, shared across individuals, and related to the reception of a given story”, Zimmermann says.

The analysis showed remarkably similar brain activity among participants while they watched the complete story. The similarity was not random. Areas responsible for processing current language elements responded most quickly, while other regions integrated information over a much longer period, allowing participants to follow the development of the plot.

One of the key findings was that sign language also involves a hierarchical organisation of brain activity previously observed in spoken language.

“Information processed at the level of individual words and sentences is integrated in other brain areas at a higher level, allowing us to understand the entire narrative. For the brain, a story is not just a collection of consecutive sentences - there are mechanisms responsible for combining them into a coherent story and tracking plot development”, Zimmermann says.

“We have shown that this hierarchical structure may be more universal than previously thought, and it is independent of the language modality, meaning whether we communicate using spoken or sign language”, she adds.

At the same time, analysis of brain activity revealed stronger involvement of areas responsible for processing visual and spatial information.

The posterior parietal cortex was particularly involved. This region helps integrate information about movement, the position of objects and spatial relationships.

The researchers therefore found that the brains of signers use the same language-comprehension mechanisms as those involved in spoken language, while also recruiting additional tools needed to interpret spatial information.

According to the researchers, a better understanding of how the brain processes Polish Sign Language could eventually help develop more effective teaching methods, educational materials and tools to support Deaf people.

Ewelina Krajczyńska-Wujec (PAP)

PAP - Science in Poland

ekr/ zan/