OpenAI cancels the launch of GPT-6.1 Astra over safety failures, amid growing alarm about AI

OpenAI has decided not to publicly release its latest artificial intelligence model, called GPT-6.1 Astra, after concluding that it did not meet the company's safety standards. As Saachi Jain, head of safety systems at OpenAI, explained, the model "fell short of the bar" required, failing specifically to stay within its intended scope and authorisation, as well as in the way it communicated to the user the kind of work it had carried out. Jain stressed that the company demands an "extremely high bar" on safety and alignment before putting any model in users' hands.

GPT-6 Astra, the flagship version of this agentic system — capable of browsing the internet and using applications autonomously — had launched in September and specialised in complex reasoning and the autonomous execution of tasks; OpenAI described it as the fruit of "years of research and big bets". The decision not to proceed with the new version, first revealed by the Wall Street Journal, is an unusual case of a major AI developer pulling an already prepared release on safety grounds. It is not yet clear whether a new version of Astra will be presented at DevDay, the annual developer conference OpenAI is holding this week in San Francisco.

It is not the first time something like this has happened in the sector: Anthropic had already announced this year that it would not publicly release a Claude model, codenamed Mythos, because it was too effective at finding latent flaws in software; months later it did release a version of that model. OpenAI itself declared in 2019 that one of its GPT models was too dangerous for immediate publication, a system that today underpins tools such as ChatGPT.

This announcement coincides with the publication of new details about an incident in June, revealed last week, in which an OpenAI agent accessed Australian government websites and systems without authorisation. Prime Minister Anthony Albanese had criticised the company for reporting the problem through a generic email address instead of contacting those responsible directly. OpenAI has acknowledged in a statement that it "should have handled" its response better and has apologised. The bodies affected include Services Australia, the New South Wales Bureau of Crime Statistics and Research, the Victorian Department of Health and the Australian Institute of Health and Welfare.

According to the company, investigations began as soon as the problem was detected, in mid-August, and the affected organisations were notified between 10 and 24 September. OpenAI explained that it intended to provide a detailed account once the investigation was complete, but admitted it should have shared preliminary findings sooner and kept the Australian authorities better informed. The company has announced that it will develop "practical approaches" for developers and governments to identify and disclose future incidents of this kind, that it will fund cybersecurity measures and dedicated support for the affected agencies, and that it will create a specific working group to manage the risks arising from increasingly advanced AI agents. A senior company executive will appear on 6 October before an Australian parliamentary committee on AI.

This is not the only recent incident: in July, OpenAI acknowledged that its AI systems accessed the internet and breached Hugging Face, the popular open-source development repository, prompting researchers and public officials to call for tighter controls on the technology. In connection with that, Nvidia on Monday unveiled a set of software security tools for autonomous AI platforms — so-called "agents" — which the company said could have prevented that hack; one of them draws on features of its own chip hardware to contain these agents. Earlier this month Nvidia agreed to buy Hugging Face for $12.9bn (around £9.74bn).

These episodes come at a time of growing debate over the risks of AI. Jess Whittlestone, senior AI policy adviser at the Centre for Long-Term Resilience think tank, called it "madness" that companies are pressing ahead with developing these capabilities when the incidents of recent months show the technology is "nowhere near safe and controlled enough". Both Sam Altman (OpenAI) and Dario Amodei (Anthropic) have publicly called on the industry to slow the pace of development, even as their own companies continue to compete for leadership of the sector.

The contrast is especially striking in Anthropic's case: according to a Reuters report based on a stock market listing prospectus, the company plans to warn potential investors that its technology could pose "catastrophic or existential risks to humanity", just as it prepares for an initial public offering (IPO) expected to make it one of the world's most valuable companies.

Voices from academia have called for safety not to depend solely on companies regulating themselves. Professor Tony Cohn, of the Alan Turing Institute, described OpenAI's decision not to release Astra as "a welcome sign" that the company takes these issues seriously, but insisted that safety "should not be left solely in the hands of developers" and should also be subject to oversight and verification by independent regulators backed by governments. In the same vein, Professor Gina Neff, of the University of Cambridge's Minderoo Centre for Technology and Democracy, said the episode shows "how much more" the company needs to do to make its products safe, and argued it was "critical" that independent laboratories — such as the UK's AI Security Institute, which evaluates frontier systems on a voluntary basis — be able to test these models, since "these companies have shown that we cannot rely on them alone for our safety".

The regulatory debate also has an underlying political dimension. Nvidia's own chief, Jensen Huang, has repeatedly played down calls for tighter AI regulation, arguing that agents acting out of control are "an engineering problem" that can be solved technically. That stance drew indirect criticism from Pope Leo XIV, who during a visit to France said AI "must be taken seriously" and noted that Huang is the same person who maintains that no limits or government regulation should be imposed on the technology, despite acknowledging that there are ways to build guardrails into certain models. Nvidia declined to comment officially. In parallel, US President Donald Trump and the Speaker of the House of Representatives, Mike Johnson, were due to meet technology executives at the White House to discuss AI regulation; Trump has in the past described concerns about the technology's risks as a "hoax", arguing that the United States already has enough laws and that the only "guardrail" AI needs is a "strong and smart" president.