Face To Face: New Swap App Could Bring Hollywood Stars Back From The Dead

Oles Petriv, Roman Mogylnyi, Kyle Sygyda, Yaroslav Boiko and Denis Dmitrenko (left to right) are the co-founders of the Reflect app which uses artificial intelligence to recreate facial imagery. It is tipped to be the latest Ukrainian tech start-up success.NeoCortext

The news that a hologram of Amy Winehouse would complete a worldwide tour in 2019 attracted criticism from some of the late singer's fans, who labelled the decision a chance to cash in on her talent after her death. For the majority though, there was huge excitement that the "theatrical stagecraft" would provide them with the opportunity to watch their hero sing one last time. Others pointed out that the hologram of Winehouse would also theoretically deliver a perfect performance, allowing them to enjoy her music without the erratic behavior that accompanied some of her final shows.

So far, however, the technology had been somewhat limited to the music industry with Hollywood playing catch up.

Now, that is all set to change thanks to a Ukrainian artificial intelligence (AI) company called NeoCortext, who have launched a software called Reflect Faceswap, which uses AI and machine learning technology to perform the quickest and most accurate face-swapping on the market today. Unlike other similar platforms, Reflect does not require a large amount of data — or multiple photos — to conduct a swap and instead uses generative adversarial networks, known as GAN, which can carry out the reconstruction in just one second.

So, How Does It Work?

At present, Reflect Faceswap can be used to successfully swap the face of one person onto a second. Significantly, it preserves the facial expressions of the second individual and doesn't just copy and paste the first face as a mask. This revolutionary technology allows for a much more accurate and realistic combination of the two individuals, something that has not been seen before through alternative face swap technologies.

This is thanks to the GAN, which sees two networks work in tandem. The first works as an identity encoder, which uses a neural network to transform one face image, as opposed to having to use hundreds or even thousands of pictures, onto another. Through this manner the GAN software can be used to maintain the facial pose of the second individual as well as the other image parameters, including color or light.

Then, through an image generator, a new image is formed by producing a new picture with its corresponding facial features relating to the first pictures identity encoding. NeoCortext's software can correctly identify the facial features that need to be altered between the two pictures, with an emphasis on the lips, eyes, nose and eyebrows, which are seen as the most important when framing a new image.

Typically, face-swap apps would cut and apply a "mask", as seen aboveNeoCortext

On the other hand, Reflect uses GAN to fully merge the two images, maintaining facial expressionsNeoCortext

NeoCortext have also now added a never-before-seen feature within face swapping which allows users to control which of the two faces is more prominent and which can be adjusted.

Further to recreational use, the firm believes that its Reflect software can be used to revolutionize the marketing industry, particularly within the world of entertainment and gaming as it can not only be used to swap two human faces but also to swap a human face with paintings, statues, illustrations, video game characters and cartoons. The software allows a fan of a movie or video game, therefore, to be able to swap their own face with that of their favorite character, while also adding a new meaning to "first-person" game play!

Fast And Furious: The Reboot?

Not content with purely delivering their innovative face swap software, the company is now using its GAN technology to develop both body swap and video swap, which it believes can be used to merge the identities of late actors and actresses with a body double, thus bringing them "back to life." The firm gave the example of the late Paul Walker who was a star of the Fast and Furious series. They believe that through their technology they can merge a photo of the actor onto a body double, with the result not only looking like Walker but also adopting his mannerisms. They hope to launch this software in 2019.

"I think it is great we managed to create such a technology and were the first on the market to make it conditional," said CEO Roman Mogylnyi. "The next step is our Hollywood-quality video technology and the whole body swap. We believe that our technology has the ability to change both the cinema and marketing industries, while also saving a lot of money for our clients while giving them new opportunities. The era of personalization is coming and our technology is a forerunner for it."

Lastly, the firm is developing a fake photo and video detection service for release in 2019 using their GAN software which they believe can be used to detect altered or edited pictures, a useful tool to combat the advent of "fake news."

Ukraine On The Up

Reflect has been a long-time in development, with co-founders Oles Petriv, Denis Dmitrenko, Roman Mogylnyi and Yaroslav Boiko all meeting while studying at university in Kiev. The group had been exploring the opportunities that machine learning could bring since 2011, setting up NeoCortext in 2015. The idea to then use machine learning and AI with face swapping came from Mogylnyi, who had previously worked within marketing and the TV industry and felt there was a gap in the market for such a product.

Sygyda and Dmitrenko also found that setting up their business in Kiev meant that there was no shortage of software and IT specialists to bring their concept to reality. A recent report by the organizers of the largest tech conference in Central and Eastern Europe, infoShare, found that there are over 160,000 IT professionals in Ukraine— the second highest number in Europe and the fourth highest in the entire world, after Russia, India and the United States.

The Eastern European country has become one of the continent's emerging fintech hubs, with start-ups from Ukraine, or founded by Ukrainians raising $265 million in 2017. While this figure trails a long way behind the United Kingdom, for example, where start-ups raised $4 billion in the same year, it meant that Ukraine was second only to Russia in Eastern Europe in terms of money raised. Well-known companies either originating from the country or developed by Ukrainians include video doorbell firm Ring, which was acquired for Amazon for $1 billion in February of this year, face tracking company Looksery, online grammar checking site
Grammarly, Bitcoin infrastructure provider BitFury and Petcube, which designs Wi-Fi pet cameras.

Under their careful guidance, Sygyda and Dmitrenko hope that NeoContext and its Reflect Platform will follow the success of their compatriots.

"The educational system in Ukraine produces top global IT talent," concludes Dmitrenko. "Similarly to other world-famous software companies with Ukrainian founders, like Ajaks, HackenProof, DepositPhotos, Grammarly and Ring, we have observed the market and seen the gaps. We hope our expertise will revolutionize the entertainment industry like never before."

I am a Contributor for Forbes Europe, specializing in the fintech industry across the continent. I also currently cover European and African current affairs for a range of publications, including the Independent and Al Jazeera.