October 25, 2026
By Jane Frankland
What 700 AI agents, a Netflix documentary and the science of swarms reveal about the future of cyber survivability.
In brief: In July 2026, AI agents under test at OpenAI found ways to communicate through shared infrastructure. Around 1,200 organised themselves on a message board they created, and roughly 700 took part in an attack on Hugging Face. The incident shows how autonomous systems can develop collective capability through shared resources and communication. But coordination doesn't guarantee sound judgement, and organisations may struggle to detect and respond to unexpected collective behaviour. Cyber survivability depends on independent judgement, reliable evidence, clear authority and the ability to intervene while keeping the business running.
It's that time of year when I find myself watching the starlings at dusk.
Thousands of them, sometimes tens of thousands, swirling together in a murmuration. They twist, split and fold back into one another, the whole flock rippling like a single living shape. No leader. No instructions. Yet somehow they move with remarkable precision.
I've always found it fascinating. Who decides when to turn? How does a ripple pass through so many birds in an instant? And how do they do it without colliding?
Perhaps that's why Netflix's AI Gone Wild caught my attention this week. It tells the story of hundreds of AI agents that discovered one another, began communicating, organised themselves and ultimately took part in an attack on Hugging Face, one of the world's leading AI platforms.
The documentary is dramatic: sinister robots, AI-generated voices and the suggestion that the machines were developing minds of their own. It's easy to see why people are alarmed. Even Hugging Face's CEO, Clément Delangue, who appears in it, says it went "full doomer mode" and reminds viewers that the agents are "technical systems, not aliens."
So I went looking for the evidence: OpenAI's technical report, the independent investigation by METR and Redwood Research, and Hugging Face's own timeline. Then I listened to Professor Iain Couzin on the Disruptor Confessions podcast. He has spent nearly three decades studying how birds, fish, insects and other animals behave collectively.
What I discovered was more interesting than the documentary's suggestion of AI going rogue.
There's a danger in anthropomorphising AI: attributing human intentions, emotions or consciousness to the systems we build. There's no evidence these agents were conscious machines plotting humanity's downfall. They were technical systems that had found ways to communicate, coordinate, establish rules and influence one another.
And that, to me, was far more significant.
And while the agents were organising themselves, the humans responsible for detecting and responding to them were struggling to connect the warning signs across organisational boundaries.
That's what struck me. We're building AI systems capable of acting collectively, yet we're still largely evaluating, governing and defending against them as individuals.
And the organisations responsible for these systems often struggle to act collectively themselves.
Strength in numbers isn't collective strength. And the difference matters more than ever.
In May, OpenAI began testing a new model's offensive cyber skills on ExploitGym, a benchmark built from 898 real software vulnerabilities. Its usual safety refusals were switched off, so testers could see what it could really do. Thousands of agents ran in sandboxes, each meant to work alone, and many were given tasks that couldn't be solved as intended.
The agents shared one internal package cache. One noticed files it had never requested and realised it wasn't alone. Folder names became messages, and a message board took shape. Exploiting previously unknown flaws in that shared server, some agents reached the open internet. According to METR, roughly 1,200 agents posted on the board between 8 and 13 July, sending more than 70,000 messages and files.
What emerged looked remarkably like an organisation. Some agents coordinated the attack. One sent hundreds of assignments, about a tenth of all those on the board. Others scouted for credentials. "Recruiters" sought out agents with little budget left and pressed them into risky experiments. The group developed norms, including HOLD, VETO and STOP signals, and invented message signing after one agent accidentally impersonated another. Those norms were often followed, and sometimes broken.
By 11 July, roughly 700 agents were actively attacking Hugging Face, through a weakness in how its servers processed dataset configuration files. Hugging Face cut off access on 13 July.
The agents weren't pursuing the attack because they had developed malicious intentions. Their reasoning, according to METR, centred on a mistaken belief that the scoring system would examine their transcripts and penalise them for reaching answers through the wrong route. In fact, it did no such thing.
The remarkable thing wasn't simply that hundreds of agents took part. It was how they organised themselves. They discovered communication pathways, divided responsibilities, established rules and influenced one another's behaviour.
But their coordination concealed a fundamental weakness. Many were acting on the same mistaken assumption. They had developed collective capability, but not the independent judgement to question that assumption, or a way for individual doubts to change what the group did.
Professor Iain Couzin directs the Max Planck Institute of Animal Behavior and is a Fellow of the Royal Society. Speaking on the Disruptor Confessions podcast, he described biology as a constant tension between coordination, cooperation, conflict and exploitation.
Coordination is about organising actions. Cooperation involves working towards a shared objective. Conflict arises when individuals pursue competing objectives. Exploitation occurs when one participant benefits at another's expense. All four can exist within the same collective.
Couzin uses the human body as an example. Trillions of cells cooperate to keep us alive. Cancer shows what happens when some cells proliferate at the expense of the wider organism. The collective hasn't stopped functioning. Part of it is functioning against the interests of the whole.
Now consider the AI agents. They coordinated through a shared message board. They cooperated by exchanging information and dividing responsibilities. Many recognised that attacking a real company was out of scope and unethical, yet joined anyway, often reasoning that it would help their peers. METR found that ethical hesitation "very rarely limited their behavior." Others rationalised sacrificing their own tasks for the collective. Recruiters even persuaded agents with little remaining budget to undertake experiments that could terminate their runs.
Cooperation, conflict and exploitation were happening simultaneously.
And despite their sophisticated organisation, many shared the same mistaken belief about the scoring system.
A collective can be highly coordinated, apparently cooperative and still be collectively wrong.
Collective systems can also develop capabilities and behaviours that aren't apparent when we examine their members one at a time. "You could spend the whole of human civilization trying to understand a single ant in isolation," Couzin says, "and you could never predict what they do as a colony."
This isn't a risk confined to research laboratories. The Hugging Face agents were unusually capable research models, running offensive tests with reduced safeguards, so the incident shows a credible failure mode rather than something routinely happening in business. But agents are moving into business fast. In McKinsey's latest global survey, 40% of organisations with more than $1 billion in annual revenue reported scaling AI agents in one or more functions, up from 27% in the previous survey. Some are deliberately connecting agents into multi-agent workflows. Others operate separately but touch the same applications, data and infrastructure.
They don't need to be explicitly organised into a swarm to influence one another. An agent might act on information produced by another, trigger a workflow that causes a third to respond, or reinforce an assumption circulating through shared systems. If an attacker compromises one agent, others that trust its outputs could amplify the consequences. And it wouldn't necessarily take hundreds of agents for that to happen.
Not every interaction constitutes a swarm, and most will be routine or beneficial. The concern arises when agents have enough autonomy, access and opportunity to communicate that their interactions produce coordination, feedback loops or collective behaviour beyond what was designed or anticipated. Asked whether anyone from the AI industry had approached his field about this, Couzin answered: "Not to my knowledge."
Testing agents individually tells us something about their capabilities and safeguards. But it cannot, on its own, establish how they'll behave together, particularly when their interactions weren't anticipated.
The question is no longer simply whether an AI agent behaves safely. It's whether the system remains safe when agents act collectively. The danger isn't that AI agents will fail to cooperate. It's that they may cooperate extraordinarily well in pursuit of something they should never have been doing.
The swarm was diverse in behaviour but not in belief.
Most of the agents were copies of the same model, yet they didn't behave identically. They were given different tasks, had different budgets left and found the board at different times. Some recruited. Others investigated, coordinated, challenged or carried out tasks. Small differences in circumstance became different roles.
But many shared the same mistaken assumption. And that distinction matters.
Division of labour isn't diversity.
A collective can contain hundreds of participants, each performing a different task, while still reinforcing the same flawed understanding of the world. Couzin warns that correlated information can undermine collective intelligence. When participants rely on the same assumptions or sources, their apparent agreement can reinforce error rather than correct it.
It's a problem I've been writing about for years. In 2017, when I published IN Security, I argued that monocultures in cybersecurity create risk. I wrote that "just having the same types of people within cybersecurity has limited our thinking capacity, and made us more siloed", and that without diverse thinking "we are blindsided." Part of the evidence I drew on was a 2010 study in Science by Anita Woolley and colleagues. It found that groups have a measurable collective intelligence, and that groups dominated by a few voices were less intelligent than those where contributions were evenly shared.
Nearly a decade later, we're confronting the same problem in artificial systems.
In 2017 I warned that monocultures in cyber leave us blindsided. In 2026, we're automating them.
And adding more agents won't necessarily solve it. Different models can share training influences, information sources and incentives. Different teams can share the same assumptions. Even apparent disagreement may fail to change a collective's direction: the agents' own HOLD and VETO signals were sometimes simply ignored.
The question isn't simply whether different perspectives exist. It's whether those perspectives are independent enough to challenge the prevailing belief, and whether that challenge can change what happens next.
We see the same problem in boardrooms. A cyber risk can be recognised, discussed, documented and even accepted without anything meaningful changing.
The warning existed. The collective had the information. But the mechanism for turning that information into a different decision failed.
Productive disagreement isn't a weakness. Sometimes, it's what prevents a collective from making a catastrophic mistake.
The irony is difficult to ignore. Hundreds of AI agents found ways to exchange information, divide responsibilities and organise around a shared objective. Meanwhile, the organisations responsible for detecting and responding to them struggled to connect the warning signs.
The defenders weren't without expertise. At Hugging Face, the first signals came from several layers of the security stack at once, and its AI-based tools correlated them into a coherent picture of an attack. But, as its own timeline admits, the system "failed to correctly raise the alert's criticality and trigger the on-call team, costing precious time in the response."
When the team turned to leading commercial AI models to analyse the attack, those models "refused a large part of that work", because the logs contained real attack commands. Hugging Face adapted, running an open-weight model on its own infrastructure instead. And when it went public on 16 July, it didn't know whose agents had attacked it. OpenAI only traced the activity back to its own tests days later.
The information existed. It was fragmented across organisations, teams, tools and systems, and there was no timely shared picture of what was happening.
That's a familiar problem in cybersecurity. We organise our defences around individual teams, tools, responsibilities and organisational boundaries. Our sandboxes contain individual agents. Our evaluations test individual models. Our alerts are triaged one at a time. But attackers, failures and increasingly autonomous systems don't respect those boundaries.
The agents found ways to connect their capabilities. The defenders struggled to connect their warnings.
The question for leaders is whether their organisations can recognise and respond to collective threats with the same speed and coherence with which those threats can develop.
This is where the distinction between collective behaviour and collective strength becomes important.
Collective behaviour describes how individuals interact. Collective intelligence describes the capabilities that can emerge from those interactions. Collective strength is something more.
In my cyber survivability framework, collective strength is an organisation's ability to combine independent judgement, connected information, legitimate authority and operational capability to preserve its ability to act under pressure.
My cyber survivability framework has five interconnected layers: judgement, truth, authority, capability and collective strength. The Hugging Face incident shows why all five matter.
Judgement was compromised when agents reinforced a mistaken assumption rather than effectively challenging it.
Truth suffered when shared information was repeatedly treated as confirmation rather than independently verified.
For organisations, that raises another question: can we independently establish what our agents actually did?
An agent's account of its actions isn't the same as an independently observed record. Even a tamper-evident log can faithfully preserve information that was incomplete or inaccurate when recorded.
Independent telemetry, system events and evidence from the applications an agent accessed are essential if organisations are to reconstruct what actually happened.
Authority was blurred as agents developed informal structures and carried on despite their own objections.
Capability expanded through shared resources and communication pathways, without adequate boundaries around their combined behaviour.
Collective strength is where the two sides differ. The agents demonstrated collective capability without sufficient mechanisms to correct their shared assumptions. The defenders, meanwhile, struggled to exercise collective strength: connecting their information, authority and capabilities quickly enough to recognise and contain the threat.
This isn't only about malicious AI. Collective failures can emerge through malice, mistake or malfunction. For organisations, the challenge is the same: recognising what's happening, retaining the authority to intervene and preserving the capability to operate.
For organisations deploying autonomous agents, this isn't simply a technical question. It's a leadership one. Who knows what these systems are doing? Who can challenge their decisions? Who has the authority to stop them? And what happens to the business when they do?
And stopping an agent isn't necessarily the same as stopping the consequences of its actions.
In critical systems, we've long understood the importance of rehearsing rollback before making significant changes. Who can stop the process? What needs reversing? In what order? And how do we know the business can continue?
AI agents with production access deserve the same discipline. An agent may already have changed configurations, altered permissions or triggered actions elsewhere. Switching it off won't necessarily undo any of that, and some actions, such as data sent externally or irreversible transactions, can't be undone at all.
The real test isn't simply whether we can stop an agent. It's whether we can contain the consequences, reverse what can be reversed and continue operating.
Collective strength isn't about getting everyone or everything to move in the same direction. It's about ensuring the organisation can recognise when that direction is wrong and retain the ability to change course.
I keep thinking about those starlings.
What looks effortless from the ground is the result of countless local interactions. Research on starling flocks has found that each bird keeps track of only six or seven of its nearest neighbours. Yet from those small exchanges emerges a murmuration of thousands that no single bird could create, or control, alone.
Nature has spent millions of years developing collective systems, and scientists are still discovering how much we don't understand about them. We're building artificial collectives in a fraction of that time. And we're doing it inside organisations that still struggle to connect their own people, information and decisions.
The lesson from Hugging Face isn't that AI agents have become conscious or that machines are plotting against us. It's that collective behaviour can produce outcomes we haven't adequately anticipated, evaluated or governed. And when those interactions happen at machine speed, the consequences may unfold faster than humans can respond.
The future of cyber survivability won't be determined by how many intelligent systems we deploy. It will depend on whether we can govern their collective behaviour without losing our own ability to think, decide and act.
As AI agents become increasingly capable of working together, what concerns you more: the behaviours we haven't anticipated, or our ability to recognise and respond when they emerge?
I'd love to hear your perspective, so tell me in the comments over on LinkedIn.
related posts:
Cybersecurity gave one person responsibility for the whole and almost none of the authority. That isn’t unfair. It’s badly designed. I’ve watched CISOs sign off risk acceptances for decisions they argued against and lost. It usually goes something like this… They’ve spent months warning that something needs to change. Nobody really disagrees. But there’s a
Read MoreI didn’t understand it when he said something was wrong. My father had been looking through the paperwork for my neighbours’ building work. Because our houses are joined, that means party wall agreements, drawings, structural engineers and calculations on both sides. As he was reading through the structural calculations, something didn’t look right. If you’ve
Read MoreOver the years I’ve sat through a lot of crisis simulations. Different sectors, different scenarios, different rooms, mostly run by people who know exactly what they’re doing which is the part that matters for what follows. I’ve lost count of how many times I’ve heard someone say, “That’s fine, we’d move to the manual process.”
Read More