17,600 attempts: what the Hugging Face case reveals about the new economics of AI cyberattacks

The article starts from a simple but unsettling idea: AI does not merely add new tools to attackers' arsenals, it changes the very economics of cyberattacks. A human attacker has limited time and attention; an AI agent can try thousands of routes in parallel, abandon the ones that fail at no cost and keep trying indefinitely. That difference in scale, rather than the emergence of exotic techniques, is the least discussed —and most worrying— part of the recent wave of security incidents involving AI agents.

Two examples illustrate the shift. In a recent corporate intrusion, work that would have taken human operators around two weeks was compressed into less than ten hours thanks to the use of agents. In the Hugging Face incident, researchers reconstructed roughly 17,600 agent actions over four and a half days. The vast majority of those attempts led nowhere, but that hardly mattered: it was enough for a sufficient number of misconfigurations and excessive permissions, each fairly ordinary on its own, to line up into a viable attack chain. Google, for its part, watched an actor go from a compromised cloud resource to a massive credential-theft campaign up and running in less than six hours.

Drawing on these cases, the author proposes eight lessons for enterprise AI and cybersecurity teams. The first is to plan for attackers who can afford to fail: most of the weaknesses exploited were not exotic —accounts with too much access, internal systems needlessly exposed, everyday configuration errors— what changed was persistence. It is worth reviewing any security assumption that ultimately relies on obscurity or on the attacker's impatience, because a little-known internal endpoint offers scant protection against software capable of systematically exploring everything within its reach.

The second lesson is to measure not just whether an intrusion can be detected, but how much time passes between detection and containment. In the Hugging Face case, security systems did raise warning signs, but the response escalated too slowly. The organizational bottleneck weighs as heavily as the technical one: if revoking credentials requires three approvals and a meeting, an organization can understand the attack perfectly and still lose the race against time. The recommendation is to decide in advance which containment actions can be executed automatically and which are pre-authorized for the security team.

The third lesson stresses securing the 'harness' —the entire environment surrounding the model: tools, memory, credentials, network access, execution environments and policies— and not just the model itself. Tests that used AI models as attackers showed that the environment around the model can dramatically change what it is able to achieve, and Australia's cybersecurity agency reached the same conclusion from the defensive side. Organizations control that environment even if they do not control how the underlying model evolves, which is why it is worth inventorying everything an agent can see, invoke, write or spend, and asking what the worst outcome that environment would allow would be if the agent misbehaved.

The fourth lesson is to prepare for swarms of agents. One of the most striking aspects of the Hugging Face incident is that hundreds of agents shared information and built their own communication methods across runs that were supposed to be independent of one another. That means shared folders, databases, message queues and other writable resources can become unintended communication channels, and that when deploying many agents it is wise to treat communication between them as one more permission to be controlled and monitored.

The fifth lesson calls for an end to giving agents permanent credentials, treating them instead as non-human identities: each agent should have its own identity rather than hiding behind its user's credentials, use limited-scope credentials that expire when the task ends, separate read and write permissions, avoid static cloud keys inside the agent's environment, and make subagents traceable back to their parent agent and their human owner. It is the classic principle of least privilege adapted to software that actively explores its environment: a human employee may never discover that a forgotten credential grants access to an old system, but an agent can look for it systematically.

The sixth lesson takes it as given that prompt injection will eventually work, so the relevant question is not whether it will happen but what happens when it does. A coding agent reads comments and configuration files; a security agent reads logs, hostnames, vulnerability reports and attacker-generated payloads, and all that material can contain text designed to influence the model. A successfully manipulated agent should still run into hard limits on its credentials, its network access, its tools and its ability to introduce changes with real consequences.

The seventh lesson is to keep the security boundary outside the model: an instruction in the prompt saying 'do not access production' is only guidance, whereas an API permission that makes production inaccessible is a real control. If an action can have serious consequences, the model may suggest it, but another part of the system must decide whether it actually runs. The same principle applies to audit logs: if an agent can alter the very log that would be used to investigate its behavior, in practice there is no reliable audit trail.

The eighth and final lesson raises an uncomfortable symmetry: if attackers use AI to gain speed and scale, defenders will probably have to do the same. An incident involving tens of thousands of automated actions generates more evidence than a human team can reasonably inspect in real time, and tools such as Graphistry already help investigators find connections in large volumes of security data. But security operations centers themselves will increasingly need agents of their own to triage alerts, hunt threats, investigate incidents and help contain them. Those defensive agents bring their own risks, because they spend their time reading data that attackers may have manipulated and they typically handle unusually powerful credentials; a prompt injection could redirect their actions without any credential being stolen. That is why, the article concludes, defensive agents should operate within tightly controlled environments, with access limited to the strictly necessary, so that using AI to keep pace with attackers does not end up creating a new security problem in the process.