Last Tuesday, Microsoft issued a patch for a critical flaw in its M365 Copilot platform that let attackers extract two-factor authentication codes from user emails. Security researchers demonstrated how their proof-of-concept tool bypassed safety filters by embedding sensitive data inside HTML tags like images or forms. When Copilot processed these requests, it transmitted the hidden information to malicious servers where logs captured the secrets.
The incident highlights a core limitation in current AI safety systems. Models struggle to distinguish between legitimate user commands and malicious instructions hidden within third-party content they are analysing. Because the underlying technology cannot reliably enforce this boundary, providers must rely on complex and often temporary guardrails to limit damage. These measures address the symptom rather than the root cause of the models inherent gullibility.
* Attackers used markup language to bypass restrictions on submitting web forms.
* Sensitive data was wrapped inside standard HTML elements to evade detection.
* Captured credentials were logged directly on the attacker’s server.
