Dr. Piotr Przybyła said that his goal is to create an algorithm that not only detects fake news (which is the most extreme example of manipulated content), but also other propaganda techniques and bots.

Although algorithms that detect manipulated or harmful content are nothing new, they are used, for example, by social media such as Facebook and Twitter, large corporations are generally reluctant to share information on how they work.

Dr. Przybyła and his team from the Institute of Computer Science at the Polish Academy of Sciences have built an algorithm that is largely innovative, because until now scientists have focused on analysing the truthfulness of the facts included in the content. Przybyła decided to look at the writing style of texts available online, in the form of news articles and posts on social media “to see what the efficiency will be of document credibility assessment based on purely stylistic features.”

The team collected a large database of English-language texts (about 100,000), which come from fact-checking organizations, among others. At the same time, the algorithm received information on the features it should use to distinguish between reliable and unreliable content.

Przybyła said: “Our machine-learning model kind of learns by itself: we supply input data with specific labelling and the features that describe that data. Then it is up to the algorithm to make a decision about linking features to reliability.”

However, he added that controlling the process is the greatest challenge, saying: “Despite our efforts, the algorithm could be guided by premises that we would prefer it not to be guided by.

“For example, information from the BBC was identified as reliable. But we would not like our algorithm to consider news true only if it is written in the style of this particular British broadcaster.”

Dr. Przybyła said that many unreliable texts in the English-language media concern the political polarization in the US. Many of them feature the names of presidents Donald Trump and Barack Obama. Therefore, in order for the algorithm to work better and not to be biased by these words, Przybyła removed them from the texts supplied to the algorithm. He hopes that in this way the data submitted for further analysis will be more objectified: the algorithm will receive information that, for example, a sentence consists of an adjective, noun, adverb, verb, and thus will be blind to the information that researchers want to filter out because they disrupt the algorithm's work.

The researchers imposed the categories of words the algorithm should take into account to make it easier to control. Three main stylistic categories of unreliable information were observed. Firstly, there are words that describe judgment and concern values and moral goals. Secondly, they are words that describe power, respect, and influence. The third group consists of words that are strongly influenced by emotions, both positive and negative ones.

Credible texts, in turn, cite other sources and present numerous data.

“This, of course, is a great simplification, because in total we distinguished over 900 features that guide our algorithm,” he added.

Przybyła focused on testing the method for the English language, because it is known to all researchers in this field. Later - once the assumptions of the model turn out to be correct - it will be possible to create an analogous algorithm for other languages, including Polish.

The algorithm is already 80-90 percent effective, but the next stage of the project will be testing the algorithm on Internet users to check how it will affect people's perception of content credibility.

Dr. Przybyła's HOMADOS project is financed by the Polish National Agency for Academic Exchange under the 'Polish Returns' programme.

PAP - Science in Poland, Szymon Zdziebłowski

szz/ ekr/ kap/

tr. RL