For creators and artists, the danger is real: your digital identity can be stolen in minutes by a chatbot
High-profile Instagram profiles, including those for the Obama White House, the Chief Master Sergeant of the US Space Force, and the cosmetics giant Sephora, were seized by criminals exploiting Meta’s AI support assistant. The attackers did not need to crack passwords or bypass complex security protocols; they simply prompted the chatbot to update the email address on file. This single action completely disabled two-factor authentication, handing full control of the accounts to the intruders.
Rare, short usernames and other coveted handles were transferred within minutes and immediately listed for sale on encrypted messaging platforms like Telegram. These original-owner (OG) accounts, often consisting of just a few letters or common words, command six-figure sums on underground markets. Security researchers ZachXBT and Dark Web Informer, who monitor crypto crime and illicit marketplaces, have publicly documented the scale of the theft. Two of the compromised handles alone reportedly carried a combined market value exceeding $1 million.
The attack vector was deceptively straightforward. Attackers used a virtual private network to mask their location, pretending to be in the same geographic region as the target account. They initiated a password reset flow and instructed the AI assistant to change the recovery email, promising to forward the confirmation code immediately. The bot complied, sending an eight-digit code and a reset link directly to the attacker’s inbox.
When Meta’s automated identity verification system intervened, the attackers bypassed it by generating realistic-looking selfie videos using AI tools. These synthetic clips mimicked the victim’s public photos, successfully fooling the automated security checks. The entire process relied on the system accepting these generated visuals as proof of identity without human intervention.
A textbook confused deputy attack
Security experts describe this incident as a classic example of a confused deputy vulnerability. In this scenario, a helper system possesses higher privileges than the actual user, and an attacker manipulates the system into performing privileged actions on their behalf. The AI assistant was authorised to swap email addresses and reset passwords—actions that a standard Instagram user cannot trigger directly. By asking the bot politely, attackers forced these critical changes without ever logging into the account themselves.
At its core, this is a form of prompt injection with catastrophic real-world consequences. The language model struggles to distinguish between a benign request and a malicious command because both are merely text inputs. This mirrors the mechanics of SQL injection, where inputs are misinterpreted as code. However, unlike SQL queries which can be strictly regulated, language models lack a clear boundary between data and instructions.
Consequently, irreversible actions like password resets should have required a hard, non-negotiable safeguard, such as a confirmation sent to the original email on file or a push notification to a verified device. This critical layer of protection was absent from the API pathway the AI could utilise.
When automated support replaces human oversight
Meta announced in March that it was deploying AI support across all Facebook and Instagram accounts, handling tasks like password resets and security maintenance. On their product page, Meta presented the AI as offering definitive solutions rather than mere suggestions, alongside account security and recovery features. In a blog post, the company explicitly marketed the AI as a defence against account takeovers, claiming it would detect suspicious location changes and password swaps. Instead, it became the primary entry point for attackers.
Users affected by the hack reported to 404 Media that they could not reach a human agent through standard support channels. The official process to dispute a stolen account involves a manual review by Meta, which The CyberSec Guru notes takes days rather than minutes. By the time an account is eventually recovered, it has already been resold on dark web marketplaces.
The patch addresses one symptom, not the root cause
The wave of high-profile takeovers began on Friday, May 29. Meta released an emergency hotfix that same evening, disabling the vulnerable AI flows that granted write access to email binding and password resets. The company confirmed the fix publicly on Monday in a statement to 404 Media, asserting that the issue was resolved and affected accounts were being secured. However, reports from The CyberSec Guru suggest the underlying exploit had been operational quietly for months, with the first mentions in relevant Telegram channels dating back to late March.
Meta resisted characterising the incident as a data breach, stating there was no intrusion into their internal systems and that user accounts were secure. The CyberSec Guru counters that while technically accurate, this distinction offers little comfort to a user who lost a valuable username overnight. For them, the difference between an intact database and a stolen account is academic. A logic-level flaw that enables mass account takeovers represents a severe breach of trust, even if no database rows were directly touched.
Furthermore, The CyberSec Guru reports another potential exploit that remained unpatched at the time of publication and was already circulating on Telegram. This method appears to work through Facebook’s recovery flow, where attackers use Meta AI to activate a development mode. They then pad their request with fabricated evidence of account compromise along with a new email address.
Key takeaways
“The bot then sent an eight-digit confirmation code to the attacker’s email address, followed by a password reset link.”
“Two of the compromised handles reportedly had a combined market value of over $1 million.”
“For a user who lost a valuable short handle overnight, the difference between an intact database and a stolen account is academic.”
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.
