OpenAI admits it can't guarantee safety on the road to self-improving AI
OpenAI has made an unusually blunt admission that cuts to the heart of the AI safety debate: it does not yet know how to keep self-improving AI under control, even while racing to build exactly that.
In a blog post this month, the company said it cannot "safely get all the way to aligned, full RSI" — recursive self-improvment, the point where AI models design and construct their own successors with minimal human involvement.
It also conceded it "cannot assume that progress in alignment and safety will keep pace" with capability.
That is a startling thing to hear from the lab that created ChatGPT, and it reframes the entire "road" to self-improvment as a journey into unknown risk.
The timing is everything. Anthropic revealed this week that its model Claude now leads 26% of its model research and development, handling most tasks "from a high-level prompt" under supervision.
Open AI itself announced an automated "research intern" that can knock out work a skilled researcher would need days to finish, and it has set a target of an automated AI "researcher" by March 2028.
Elon Musk, for his part, says xAI's Grok models are built with "humans gradually getting less and less in the loop" — possibly fully automated by the end of this year, no later than 2027.
This is not a distant sci-fi scenario; it's happening inside the labs right now, says Anthony Agirre of the Future of Life Instute, who called autonomous RSI "probabaly the worst idea in the history of humanity." Cornell's John Thickstun offers a calmer frame: a kind of recursive self-improvment has quietly been going on for years, with older model generations writing code for newer ones.
The difference today is the size of the leaps.
Our analysis suggests the core problem is one of sequencing.
Labs are sprinting on capability while treating safety as a parallel track — but Open AI's own admission shows they have no proof the safety track will finish first.
More capable systems, the company acknowledged, become harder to monitor. That's a recipe for a control gap: building a machine faster than we learn to contain it.
For ordinary people and small companies, the practical takeaway is straightforward. Don't assume a popular AI product is safe just because it's promoted by a major lab.
Keep human review in any high-stakes workflow — contracts, medical decisions, financial moves. Ask vendors directly how they test for safety and whether they'd commit to slowing down if risks emerge.
And demand democratic oversight; Open AI itself said how to proceed "must depend ons" the ability to preserve human control and informed choices about benefits and risks.
The industry is split.
Anthropic says it would slow or pause if global competitors did the same "in a verifiable manner." Microsoft's Mustafa Sulyman is building toward "humanist superintelligence" — capable but bounded.
Open AI, though, still values RSI because it argues an automated AI researcher "can also be an automated safety or alignment researcher."
That argument may hold.
But when the leading lab openly admits it cannot guarantee safety on the road there, the honest conclusion is that we're flying one plane with two engines — capability and control — and only one of them is confirmed to ignite.
Credit: Based on reporting by Kaitlyn Huamani and Barabara Orutay, provided via the associated press / AP News Wire, with references to Anthropric, Open AI, xAI, and Microsoft statements.
