OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM

OpenAI has launched a new image classifier and updated ChatGPT safety measures to address child sexual abuse material (CSAM). The measures include…

By Vane November 16, 2026 2 min read
OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM

OpenAI has launched a new image classifier and updated ChatGPT safety measures to address child sexual abuse material (CSAM). The measures include a detection tool for images generated by AI, warning screens for users, developer guidelines, and specific resources for law enforcement.

New detection tool

The company unveiled a classifier capable of identifying CSAM images, including those created by artificial intelligence. Selected partners can access this tool via the API without a fee. The system scans for visual indicators of abuse not previously encountered and provides a score to guide reviewers on how to handle flagged content.

OpenAI developed the classifier with the National Center for Missing and Exploited Children (NCMEC). The firm is collaborating with the Tech Coalition and ROOST to distribute access to the software. Early testing with partners showed the tool flags 48% fewer images while maintaining the same detection rate for abusive material. This reduction lowers the volume of work for human reviewers. Images flagged by the system are twice as likely to contain abuse compared to unflagged content.

The company is introducing warning screens within ChatGPT to stop users from accessing or creating content that sexualises children. The interface displays an alert to anyone attempting such prompts. It directs users to specialised support services aimed at stopping harmful behaviour. Accounts attempting these actions face a 24-hour suspension. The application already blocks CSAM and related requests. Repeated offences may result in permanent account removal.

Guidelines and partnerships

OpenAI formulated these messaging and guidelines in consultation with organisations and researchers. Partners included Protect Children, the Lucy Faithfull Foundation, and Joel Scanlan from the University of Tasmania. Scanlan independently evaluated a trial of the deterrent messages.

The firm is publishing guidance for developers using OpenAI tools to prevent, detect, and report CSAM. Early testing suggests these guidelines have encouraged developers to adopt child safety tooling and register with NCMEC to report abuse. Law enforcement agencies dealing with child safety receive AI-powered features, including dedicated plugins, investigative tools, and complex data analysis. Access is managed by programme administrators. Investigators also receive training and a field guide on using AI.

Context and history

OpenAI faces several lawsuits alleging its technology harms users, particularly children. Safety-focused nonprofit Common Sense Media categorised OpenAI’s technology as an unacceptable risk for teenagers. Other AI chatbots, including Grok, have been used to create nonconsensual sexualised imagery, including images of children.

The startup implemented changes over the last 12 months. These steps included releasing a safety blueprint, updating content guidelines to prohibit inappropriate material, retuning models to prevent harmful generation, and launching a special version of ChatGPT for teens.

What it means

Developers using OpenAI’s API gain free access to a new filter that reduces false positives on abusive images, cutting manual review work in half. Users attempting to generate or view CSAM face immediate warnings, links to support services, and a one-day account suspension. Law enforcement teams get better tools to analyse data, while safety researchers helped define the rules for these new warnings.

Scroll to Top