Anthropic has updated its S-1A filing to state it is nearing the point where its models can recursively self-improve.
The company disclosed this warning alongside financial results showing rising revenue and increased expenses. The new text appears in the fine print of the document.
What is recursive self-improvement
RSI describes a stage where an AI system trains itself to become better at training itself. This creates a rapid acceleration in capability.
Some observers view this as a point of no return where machines build themselves beyond human control. Others see it as a way to achieve high productivity and scientific breakthroughs.
Anthropic treats the concept as both a risk and an opportunity. The filing notes that using AI to build AI carries safety risks, especially if progress speeds up quickly. This falls under the company’s Responsible Scaling Policy.
Warnings in the filing
The company added statements to its Risk Factors section earlier this year. It stated that it has already used AI to create models with capability jumps it did not recognise at the time.
Anthropic now says it may soon meet its own definition of RSI. The text reads:
In addition, our increasing use of AI systems to accelerate our own research and development may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures. As a result, we have in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred.
For example, our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of “dramatically accelerated progress attributable to AI.” We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data.
Keeping humans in the loop
The filing confirms that none of Anthropic’s current models have crossed the RSI threshold yet. The company promises to keep humans involved in research and development for the foreseeable future.
Anthropic stated it does not expect to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding. It is considering updates to its Responsible Scaling Policy to maintain human control over model development.
Reality check
Critics may argue that an S-1 filing serves as both disclosure and marketing. Claims about being close to a self-building model could be promotional.
However, Claude’s capabilities in coding, mathematics, and cybersecurity have grown significantly this year. If Anthropic believes RSI is near, others should consider acting and governing accordingly.
