HELLO BOSS
“Oh my god. Oh my god,” I yelled as I looked at my own face on someone else’s body. It was all there: my five o’clock shadow, my goofy grin, even the bags under my eyes.
What it means for makers and artists
I was on a Microsoft Teams call interacting with this deepfake version of myself in realtime. Ordinarily the other person on the line looks nothing like me, but by using a gaming laptop and a sought-after, cutting edge piece of software for scammers, his face morphed into mine. My deepfake pinched his cheek, covered his nose, and stroked his chin, all without the illusion breaking.
- Whereas video deepfakes used to be about superimposing someone’s face onto an existing video, the tool I was using promised something else: the ability to shapeshift into someone—anyone—in real time during a video call. After weeks of back and forth with the Chinese-language scammers selling the tool, called Haotian AI, I had obtained a copy of the software.
- Haotian AI is built to work specifically with platforms we all use every day: WhatsApp, Microsoft Teams, Zoom, TikTok, Instagram, and YouTube. It marks the next stage in deepfake scams and fraud, where criminals are able to change their appearance in real time to trick people, including Americans, into handing over their money.
- Our investigation finds Haotian AI demonstrates its tool as a way to impersonate at least one U.S. police department. We link it to Chinese money laundering networks and the ecosystem providing services to massive scam compounds in South East Asia, and find that Haotian AI has brought in more than $4 million dollars for its creators.
- Haotian AI is likely based on open source face swap tools, meaning the true value of the software lies in its sophisticated technical support. With that, even the least tech-savvy criminals can now access realtime deepfake software, opening up the possibility for more fraudsters around the world to use this powerful technology.
Haotian AI’s realtime deepfakes can be particularly impressive, able to handle adjustments in lighting and objects appearing in front of the subject’s face. According to demos the company has posted on Telegram, one demo video shows an Asian woman magically transforming into actor Gal Gadot. In the demo, the deepfake Gadot blows a kiss, covers one of her eyes, and rapidly swipes her hand past her face, with the software not glitching once.
Other tools sometimes fail when a user touches their own face. When they do so, the software malfunctions and shows the real person underneath. In the demos, Haotian AI keeps the illusion going, though. Another demo video shows a realistic, albeit dewy skinned, Elon Musk and Jackie Chan.
In a direct and live demonstration with me over the messaging app Telegram, a Haotian AI technician showed in real time how the software can also make a subject’s lips thicker or thinner, or their jawline sharper or more rounded. The user interface of Haotian AI also lets customers adjust the size of their deepfake’s nose, use an “acne removal” feature, and change the shape of their eyes.
For an effective deepfake, some configuration may be required, according to 404 Media’s own tests. And Haotian AI may not just beat the human eye but tools designed to detect deepfakes as well. Xception is a deepfake detection model; in a paper published last June, researchers found it “struggled” to detect Haotian AI-generated deepfakes.
“While Xception hit 89.1% accuracy on the control stuff, it misclassified almost 100% of the Haotian samples as ‘authentic,’” Charles Fross, one of the authors of that paper, told 404 Media in an email. “Haotian’s work is shockingly convincing, especially with how it handles facial and body movements.”
There are now a variety of ways to produce deepfakes, but generally they work by training a machine learning model on images of a person’s face, then mapping that face onto another face in a video, frame by frame, a process that could take seconds, minutes, or more, depending on the method. Realtime deepfakes work similarly, but are more sophisticated because they have to track a face and map the deepfake face onto it moment to moment.
Key Takeaways
- Haotian AI demonstrates its tool as a way to impersonate at least one U.S. police department.
- The software is likely based on open source face swap tools, meaning even the least tech-savvy criminals can now access realtime deepfake software.
- Xception struggled to detect Haotian AI-generated deepfakes, indicating its work is shockingly convincing.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.
