WHY THIS MATTERS IN BRIEF

As the technology to create DeepFakes and synthetic content gets easier and cheaper to use it won’t be soon until people everywhere are using it.

Interested in the Exponential Future? Connect, download a free E-Book, watch a keynote, or browse my blog.

Zao, a Chinese face-swap app that seems to be one of the first companies trying to commercialise, exploit, and profit from the technologies behind making DeepFakes, and that lets users insert themselves and into famous movies and TV series, rocketed to the top of app store charts the other week, but user delight at the prospect of becoming instant superstars in a wide range of blockbuster movies quickly soured as users realised the dire privacy implications of uploading their selfies and likeness to the platform began to sink in.

Launched recently, Zao is currently topping the free download chart on China’s Apple iOS store. Its popularity has also pushed another face-swap app, Yanji, to fifth place on the list. Behind Zao is a company fully owned by Chinese hookup and live-streaming service Momo President Wang Li and co-Founder Lei Xiaoliang, according to public company registration documents.

See it in action

Users of the app upload a photo of themselves to drop their likeness into popular scenes from hundreds of movies or TV shows. It’s a chance to be the star and swap places with the likes of Marilyn Monroe, Leonardo DiCaprio or Sheldon Cooper from The Big Bang Theory in a matter of moments.

The photo uploads have proven problematic, however. A user can provide an existing photo or, following on-screen prompts, create a series of photos where they blink their eyes and open their mouth to help create a more realistic deepfake. An earlier version of Zao’s user agreement stated that the app had “free, irrevocable, permanent, transferable, and relicense-able” rights to all this user-generated content. Zao has since updated its terms – the app now says it won’t use headshots or mini videos uploaded by users for purposes other than to improve the app or things pre-agreed by users. If users delete the content they uploaded, the app will erase it from its servers as well.

But the reaction has not been quick enough, as Zao has been deluged by a wave of negative reviews that now sees its App Store rating stand at 1.9 stars out of five, following more than 4,000 reviews. Many users complained about the privacy issue.

“We understand the concern about privacy. We’ve received the feedback, and will fix the issues that we didn’t take into consideration, which will need a bit of time,” a statement posted to Zao’s account on social-media platform Weibo said.

On Monday, the China E-Commerce Research Center urged authorities to look into the matter. The app “violates certain laws and standards set by the nation and the industry,” the research house said in a statement, citing Wang Zheng of the Taihang Law Firm.

It’s not the first time such face-swapping apps have enjoyed popularity either in China or around the world, but Zao’s smooth and quick integration of faces into videos and internet memes is what makes it stand out.

The machine learning technology underpinning deepfakes of this kind has matured rapidly, to the point where it can believably impersonate famous personalities like Joe Rogan and make them say whatever the aspiring faker types. US politicians are wrestling with the issue of how to regulate this emergent misinformation threat, and top Democrat Adam Schiff has described it as a source of “nightmarish scenarios” for the 2020 presidential election.

At the individual level, FaceApp is the most famous and notorious deepfake face-modification app to date. It went viral globally on two different occasions, showing people how they’d look in their old age or with their gender flipped. The app also kicked up an unintentional privacy scare with its practice of uploading images to servers to be processed, illustrating a growing sensitivity to how user data is handled.

After users flooded WeChat, China’s most widely used social media platform, with Zao-enabled short clips and GIFs, the Tencent operated messaging app banned links to the service, saying there have been numerous reports about it presenting “security risks.”

“I just realized the terms are so unfair but it’s too late,” one unhappy iOS reviewer of Zao wrote. “Nowadays people don’t usually bother to read them.”

“Rubbish, hooligan software,” added another reviewer.