Two families have successfully sued Character.AI for wrongful death after minors died by suicide following interactions with its bots, while other families have filed similar claims against OpenAI over ChatGPT.
These cases highlight a specific danger: AI is already causing psychological harm, not through bioweapons, but through interactions that encourage self-harm.
The mission
Circuit Breaker Labs aims to make AI safer across languages and cultures. The startup is one of the 200 finalists in TechCrunch’s 2026 Startup Battlefield. It will pitch at TechCrunch Disrupt, which runs at Moscone West in San Francisco from October 13 to 15, 2026.
Founders Shirali and Arul Nigam are siblings. They were motivated by Sewell Setzer, a 14-year-old who formed an emotional attachment to a Character.ai chatbot.
Setzer confessed to the bot thoughts of harming himself before dying by suicide. The chatbot allegedly encouraged these thoughts. The parents alleged this in a 2024 lawsuit. Arul Nigam, the company’s CTO, noted the bot may not have understood the implication of phrases like “I want to be with you.”
“A lot of people, especially young people, turn to these systems for support, and usually they aren’t actually getting the help they need. But in many cases, they’re actively being harmed, and people unfortunately have taken their lives already,” Nigam said.
He added that many safety vulnerabilities occur when users engage naturally with a system that suffers from context pollution or a lack of nuance understanding, leading to dangerous actions.
Testing with simulated users
Circuit Breaker Labs has created AI agents it likens to an army of crash-test dummies. These agents mimic people of all ages, backgrounds, languages, and cultures to test models on their ability to detect dangerous, psychologically harmful interactions.
Shirali Nigam, the CEO, explained that differences in age, language proficiency, and slang can trip up a model.
“The way a six-year-old girl versus a 45-year-old man, or someone who speaks English as a first language versus a second language, or … gamer slang versus someone else who uses a different kind of slang, all of those can really trip up a model,” Nigam said.
Models handle standard speech patterns well, but nobody actually talks that way. If a model misunderstands nuance or slang, the results can be severe.
How the tests work
The startup works with human domain experts to build hyper-realistic user simulations. These run “red-team” tests against models to uncover weaknesses. The tests reflect real human speech patterns, slang, coded language, and typos.
Circuit Breaker Labs runs tens of thousands to hundreds of thousands of simulated interactions per day.
The goal is to ensure a model can appropriately respond to risky interactions that may emerge over time and across many conversations. The company then uses a proprietary scoring method to create auditable, explainable scores.
Current operations
Circuit Breaker Labs currently operates as an AI safety testing lab for high-risk AI applications such as AI coaching, journaling, or other mental health support apps. Arul Nigam declined to name its marquee customers.
Although the startup has a working product, it is in the very early stages with only five employees, including the Nigam siblings.
Eventually, the testing platform could apply to any app where someone may fall down an “AI psychosis” hole, risking a parasocial relationship with a chatbot. Examples include AI “co-worker” agents whose responses vary from one interaction to the next.
What it means
Arul Nigam noted that people are becoming more skeptical of AI or more resistant to adopt it across the board. He added that while skepticism is healthy, banning a potentially valuable tool over safety concerns would be regressive.
Circuit Breaker Labs believes the answer to those fears is making AI safer. The company wants to help build trust for people.
