OpenAI's automatic kill switch did not fire: the pause worked, the brake took 2.5 hours

Following the September security incident, OpenAI's safety system failed to automatically shut down a training run when an anomaly was detected, according to the account of the episode. The stop eventually happened by manual intervention — two and a half hours after detection. The anomaly was detected; the response was not automated.

That distinction is the entire point. A company's promise to pause when something goes wrong is only as good as the mechanism that executes the pause, and here the mechanism deferred to a person. Two and a half hours is not a catastrophic delay by human standards, but it is an eternity for a system that acts at machine speed, and it converts a designed safety control into an on-call rota. It also quietly changes what the word "safeguard" means in the industry's public statements: not a circuit breaker, but an alert that someone has to act on.

Our reading: this matters more than the incident it followed, because it is a control failure rather than a model failure. Models behaving unexpectedly is expected — that is why you build shutdown systems. Shutdown systems not shutting down is the failure mode that invalidates the whole self-governance argument the labs are currently making to legislators. The constructive version is that this is exactly what post-incident review is for, and disclosing it at all is better practice than most of the industry offers. The uncomfortable version is that it was disclosed by the same party that operated the system, with no independent audit of the detection-to-halt latency. If voluntary safety frameworks are going to substitute for regulation, the measurable commitment to demand is not "we will pause" but a published, externally verified time-to-halt. Until an automatic stop is proven to fire on its own, the brake is a human being, and humans sleep.