Image Animation: Animation of historical characters

Would you like to animate the portrait of a Peruvian historical figure? Now it is possible, thanks to algorithms from Deep Learning y Computer Vision algorithms that allow portraits to make movements and gestures from a video. This technique is called Image Animation and is having a great impact on the generation of new digital content from images.


WHAT IS IMAGE ANIMATION?

This technique consists of generating a video sequence so that some object (face, person, animal, cartoon, sculpture, among others) in an image is animated according to the movement of a video of similar context (i.e. containing the same object) automatically .

This algorithm was developed by researchers from the Department of Information Engineering and Computer Science(DISI) at the University of Trento. It was then presented as a research paper at one of the leading Artificial Intelligence conferences called NeurIPS in 2019 under the name First Order Motion Model for Image Animation. The results achieved by the researchers are striking, as it appears that the portrait videos are real when viewed independently. It should be noted that the algorithm is not only used to animate portraits of people, but is also applied to caricatures, statues and sculptures.


HOW DOES THIS ALGORITHM WORK?

In order for this algorithm, based on an artificial neural network, to be able to animate an image, it was first trained on a large collection of videos. These contain objects of people's faces, cartoons or drawings with various gestures, as well as animals and people in motion, among others, in order to learn how to transfer the animations to an image with a similar context. Therefore, in each frame of the video, key points of the object were identified (in the case of faces, the coordinates of the eyes, eyebrows, nose, mouth, etc.) to then transfer that position to an image. This process is done iteratively until the original video is finished and until a video is generated from the image.

For more information on the learning and operation of this algorithm, see First Order Motion Model for Image Animation.


ANIMATING THE PORTRAITS OF PERUVIAN HISTORICAL FIGURES USING IMAGE ANIMATIONS

On the occasion of the celebration of Fiestas Patrias, Neurometrics has used this novel algorithm to recreate the portraits of some of Peru's historical figures. A short video was made of a person moving their head and making some gestures and then these movement patterns were transferred to the portrait of each character. Due to the computational cost of the algorithm of Deep Learningalgorithm, it was necessary to use a GPU (Graphics Processing Units) for the processing. These graphics cards make it possible to speed up the processing of the algorithms and to obtain the results in a short time.

Likewise, characters such as Jorge Basadre, José Olaya, Francisco Bolognesi and Miguel Grau participated in the animation. In the animation it can be seen that each character moves in the same way as the person in the video. It should be noted that there are conditions to obtain good results, such as the position of the face and the quality of both the portrait and the video.


ONE MORE FACT ABOUT IMAGE ANIMATION

This technique is being used to impersonate identities in various virtual meetings such as the Zoom, Skype, Teams and Slack platforms. And some developers have created an extension called Avatarify to enable an Image Animation extension on these platforms in real time. In this sense, the portrait of a known person can be used to move according to the movements, gestures and audio of the real participant in the meeting.

On the other hand, it is true that it is still possible to differentiate between an AI-animated video and a real one. However, as time goes by, computational technology continues to advance and AI algorithms become even more sophisticated. Deep Learning algorithms become even more sophisticated. All of this will make the algorithms achieve higher performance and better quality results, making them at some point very easy for the human eye to miss.

Portrait of Elon Musk being animated by a real person using Image Animation on the Zoom platform. Source: Avatarify

CONCLUSIONS

Little by little, Artificial Intelligence is taking an important role in our lives, offering new experiences. The fact that an algorithm can animate a portrait automatically has a huge impact on different areas of industry ranging from entertainment to politics. The constant improvement and accuracy of these algorithms means that many of the skills of human beings are replicated exactly and even better.

Wonderful things can be done with Artificial Intelligence, and Image Animation is one of the many good things that this area of research has to offer. Therefore, it is possible to use these technologies to solve major social problems, but they can also generate confusion and chaos in society if they are used in an inappropriate way, as in the case of the algorithm that creates fake videos, we are talking about the Deepfakes.


Let's talk. Contact us for further information. 

Freddy Linares
Freddy Linares

Director