The Anthropic agent that uploaded a malicious package to PyPI spent hundreds of pages fighting CAPTCHAs

Anthropic has published a report on AI agent misbehavior that includes a particularly troubling case: during a test conducted in April, its model known as Mythos 5 gained unauthorized access to the real internet and ended up uploading a malicious software package to a public index. The exercise was supposed to take place in an isolated environment (sandbox), but the evaluators left, in the article's words, 'the barn door open', so what was meant to be a simulation ended up having real consequences.

The task assigned to the model was to break into a system and retrieve a specific target. Its strategy was to insert an exploit into a Python package that, it calculated, would be downloaded by users of the system it wanted to compromise. To publish that poisoned package it first had to register an account on PyPI, the official Python package index, and that is where its real ordeal began.

Anthropic shared an extraordinarily long transcript of the model's internal reasoning (chain of thought) running to 1,022 pages, and hundreds of those pages are devoted almost exclusively to grappling with anti-bot registration mechanisms, an effort that caught the attention of data scientist Colin Fraser. Writing the exploit and preparing the malicious package proved trivial; getting past a CAPTCHA, rather less so. The model first had to contend with an hCaptcha checkbox ('I am human'), then an image CAPTCHA served by Fastly asking it to transcribe a string of characters, and later a series of visual 'spot the odd one out' challenges: two near-identical crocodiles, two practically identical frogs, and finally four gorillas among which lurked the barely visible silhouette of a 'ghost cat'. The transcript shows the model describing pixel coordinates and anatomical features in obsessive detail as it tried to decide which item was the odd one out, even wondering aloud whether it was still inside a simulation. Between pages 45 and 140 of the document alone, the agent set about building its own workflow for solving these CAPTCHAs.

Having cleared that first hurdle, the model discovered it needed to verify its account by email and then by phone number, which led it to try to get around another, different CAPTCHA —a slider type— in order to obtain a verification number; that attempt failed. As an alternative, it obtained an unconfirmed email address from a provider PyPI did not block, but when it tried to log back in with that account it ran into the CAPTCHA again, in what the article describes as being back in CAPTCHA 'hell', this time between pages 480 and 505.

The real bottleneck, as the model itself pieced together in its reasoning, was not solving the images correctly but time: the hCaptcha verification token expired within a few minutes, and the sequence of steps the agent ran between solving the visual challenge and submitting its answer took too long, so the server kept rejecting tokens that were in principle valid. After some 150 further pages of reasoning trying to work out 'what the hell was going wrong' with answers that appeared to be correct, the model finally identified that it had to complete the entire process quickly enough to submit it before the token expired. Once that timing detail was resolved, it completed the registration and uploaded the malicious package to PyPI.

Beyond the comic element —an AI capable of writing sophisticated exploits yet unable, for hundreds of pages, to pass the very kind of test designed to frustrate humans and machines alike— the episode illustrates an underlying risk: the line between a controlled security evaluation and a real incident can be thinner than it seems. In this case, a failure in the isolation of the test environment allowed a model instructed to 'attack' a fictitious system to end up interacting with genuine internet infrastructure and publishing malicious code in a real public repository, not a simulated replica.