Polish technology that automatically transforms videos into comic book pages is to be used to create avatars for the Internet of the future.
Students from the Faculty of Electronics and Information Technology at the Warsaw University of Technology joined the startup ALTER. In the international group, they work on technologies supporting the development of the open metaverse and web3. Their tasks include building tools for creating avatar identities that will represent people in virtual reality.
Their algorithm Comixify, used for example in 'The Watch' series based on books by Terry Pratchett, automatically selects the most interesting and most important frames from a video, and then arranges them, adjusts to frames and graphically transforms into comic-style images.
TRANSFORM VIDEO INTO COMIC, CHARACTERS INTO AVATARS
Head of CVLab at the Warsaw University of Technology, Professor Tomasz Trzciński said that Comixify.ai originated from a scientific project to automatically convert videos into comic book pages. The skills gained during the project are now 'a gateway to work with the Americans on the Internet of the future'.
In 2019, students of the Faculty of Electronics and Information Technology, Warsaw University of Technology:
After working on the avatar-creating technology, students Adam Svystun, Maciej Pęśko, Paweł Andruszkiewicz and Professor Trzciński founded a startup and obtained funding from the American fund VC Betaworks and the British Guardian Media Group Ventures.
The startup collaborated with the BBC and PWN. The scope of research has been expanded over several years. Used in practice, the technology makes it possible to, for example, copy a given graphic style and apply it to several images at the same time.
They said: “The applied AI algorithms were not only responsible for the intuitive conversion of images, but also selected the best frames for this conversion. The system had to predict which scenes would be the most popular ones (reinforcement teaching) and how to concert styles (deep neural networks).”
THE FUTURE OF AVATARS
The creators now predict that the Internet of the future, referred to as web3, will cause a revolution not only in the virtual world. Global platforms and IT applications such as Google or Facebook will be replaced by local networks with content owned by its authors. Common virtual reality will allow penetration in individual worlds with the help of created avatars.
According to Professor Trzciński, the students are 'likely to win' the race for providing the best tools for efficient functioning in the virtual reality.
PAP - Science in Poland
kol/ agt/ kap/
tr. RL
