The Crash-Test Problem & Why Safer Software Won’t Come From Goodwill 

Last week I argued that AI has just handed our industry an enormous safety dividend, and that risk homeostasis warns us we will quietly spend almost all of it on speed unless we deliberately choose otherwise. I ended by asking you what is really stopping more teams from making the other trade, spending some of that windfall on safety and quality instead of pouring it all back into velocity.

The conversation that followed was insightful. But each comment, I kept noticing the same assumption sitting underneath the best of them, mine included, that the trade is a choice. That if we could just persuade enough engineers, enough leaders, enough boards to be a little more disciplined, the dividend would find its own way to safety.

I’ve spent 30-years in this industry, and I no longer believe that. Not because people are reckless, but because we keep asking individuals to behave well inside a system that rewards the opposite. That, not capability and not character, is the layer underneath the layer.

The Lesson Hiding in the Car, Again

I keep returning to the car because it keeps being right. Last week’s point was that anti-lock brakes didn’t reduce accidents, because drivers spent the new safety margin on going faster. The technology worked. The behaviour adapted. The benefit evaporated.

So here is the obvious follow-up question. If drivers wouldn’t bank the safety gain voluntarily, how did cars ever actually get safer? Because they did. Deaths per mile have fallen dramatically over decades.

The answer is that we stopped relying on the driver’s good intentions and changed the system around the driver instead. Three things did the heavy lifting, and not one of them was an appeal to virtue:

1. 1. We made safety visible, with crash-test ratings buyers could compare.

1. 1. We made the maker accountable, through product liability and the cases (see Ralph Nader, and the Ford Pinto) that ended the era of shrugging at predictable harm.

1. 1. And we set a floor, through standards and seatbelt laws that took the cheapest, most dangerous corners off the table entirely.

None of that asked anyone to be a hero. It rearranged the conditions so that the safe choice became the rational one. That is the same instinct I opened this whole series with – don’t chase better behaviour, change the conditions around the problem.

Security Is a Credence Good, and That Is the Whole Problem

A peer, Jen Easterly, reminded me recently in a post that software security is a textbook credence good. It is a thing whose quality you cannot assess even after you have bought it and used it for years. You are trusting that the work underneath was good, and you have almost no way to verify it yourself. Medicine, car repairs, financial advice and structural engineering all live here.

Cars used to live here too. For decades buyers could not see safety, so the market rewarded the things they could see – chrome, horsepower, styling. Then crash-test ratings put a number on the windscreen, safety became legible and comparable, and the market quietly reorganised itself around the thing it could finally judge. Safer cars started to sell. Manufacturers started to compete on the score.

Software is still sitting in the pre-rating era. The quality of the code is invisible to the people who depend on it, so we reward features, convenience, speed-to-market, and price exactly the visible attributes, and we underprice the invisible one. This is precisely why exhortation does not work. You cannot will a market into rewarding something it structurally cannot see.

“Please value our security” is not a strategy when the buyer has no instrument to detect it!

The Find-Versus-Fix Gap is an Accountability Gap

This is where last week’s most uncomfortable number comes back. Glasswing’s partners surfaced more than 10,000 confirmed high and critical flaws across the open-source commons, and only a few dozen were actually patched. I called that the anti-lock-brake effect written in code: a huge new margin, spent almost entirely on finding faster rather than fixing.

But look one level down and the gap is not really about capability. Finding is now nearly free. Fixing requires something finding does not – an owner. Someone whose job it is, whose budget it is, whose name is against the outcome. And across the software everyone else is built on, that person frequently does not exist.

There is a famous cartoon of all modern digital infrastructure drawn as a teetering tower, the whole thing resting on one small project maintained, thanklessly, by one person somewhere unglamorous. We laughed at it, then we lived it through Log4Shell and the near-misses since. AI has now handed us a floodlight that lights up 10,000 cracks in that shared foundation, and in the same beam it reveals the harder truth: the foundation has no landlord. The gap between flaws found and flaws fixed is not an engineering gap. It is an accountability vacuum.

This Is Not a Call for More Compliance

Here is the trap I’m determined to avoid, the same way I refused last week to turn this into a plea to slow down. The lazy conclusion is “regulate it.” But regulation done badly produces exactly the failure I keep warning about – performed cyber resilience rather than built cyber resilience. Checkbox security. Audit theatre. A wall of green ticks that no threat actor has ever respected.

The purpose of accountability is not paperwork. It’s to do the two specific jobs the car did:

1. 1. Make the invisible visible, and

1. 1. Give the risk an owner.

Done well, it changes what the market rewards, so security stops being the expensive, heroic option and becomes the default rational one. Done badly, it becomes one more framework nobody believes in, draining energy from the work that actually reduces risk.

So the test for any mechanism we reach for is simple. Does it make security legible and owned, or does it just generate evidence of effort?

What the Levers Actually Look Like

The software equivalents of crash ratings, liability and standards are no longer hypothetical. In Europe especially, several have moved from idea to binding law on a 2026 and 2027 timetable. The direction of travel is now concrete enough to plan around.

Visibility. Software bills of materials, the idea of an “ingredients label” for what is actually inside a product, and emerging ways to make security posture legible to buyers and boards. This is the crash-test rating, and it is the lever I would pull first, because it attacks the credence-good problem at the root. And it isn’t only a cyber idea. Next door in online safety, Australia’s eSafety Commissioner, Julie Inman Grant PSM, built the Safety by Design framework on the same logic, that the burden of safety shouldn’t fall on the user and that transparency is a hallmark of the real thing, and when platforms wouldn’t reveal what they were doing about the gravest harms, she was given powers to compel disclosure. Different harm, identical move: you cannot fix what the market cannot see.

Liability. The slow, hard shift away from disclaiming all responsibility in the licence agreement and towards makers owning the harm they ship. The US named this direction in its 2023 cyber strategy; Europe has since made it concrete. From December 2026 the EU’s revised Product Liability Directive treats software, including standalone software and AI systems, as a product subject to no-fault liability, and bars makers from contracting out of it in the terms of service. It is the Pinto lesson, decades late.

An MOT, not just a seatbelt. As I argued in the first piece of this blog series, we don’t let a car on the road on the promise it won’t crash; we make it pass an MOT, and technology is finally being held to the same test. The EU’s Cyber Resilience Act is the clearest version: from 2027, products with digital elements must meet secure-by-design requirements to be sold in Europe at all, and they must keep handling and reporting vulnerabilities across their supported life, with those duties beginning September 2026.

That is the closest thing yet to a cyber MOT: a standard you must pass before you trade, plus an obligation to keep handling vulnerabilities while the product is supported. The UK took an early step the same way, with its Product Security and Telecommunications Infrastructure (Product Security) law banning default passwords and forcing makers to declare how long they’ll support a device. Security becomes a condition of market access, not an optional differentiator, a standard you meet before you trade rather than a fine you pay after you’re breached.

Funding the commons. Someone has to pay the maintainer in the cartoon. The fix gap will not close on volunteer evenings and goodwill, and pretending otherwise is how the next Log4Shell gets written today.

And none of this is anti-speed, which is the objection I always get. Crash ratings did not slow cars down. Safer cars got faster. A floor under the worst behaviour does not lower the ceiling on the best. Brakes, as I said last week, are what let you go fast in the first place.

I’ll be honest about how far this gets us, because it’s less far than the levers make it sound. Everything above builds a safer car – fewer flaws shipped, an accountable maker, a product that must pass its MOT. But a safe road was never just well-built cars. It needed drivers who’d passed a test before they were let loose at all. In an organisation that driver is leadership – the board and the chief executive, competent enough to own cyber rather than hand it to a risk register or a lone CISO and call it covered.

That competence sits at the base of the whole resilience hierarchy. The specialists doing the hands-on work need proven competence too, which is why practitioner certifications exist, but the wheel is held at the top, and increasingly it’s the people holding it who answer for the crash. It needed crumple zones and airbags for the crashes that happened anyway, the detection, containment and recovery I spent the first piece on. And it needed the duty to drive responsibly, which binds the operator, not the maker, and sits in a separate regime, the EU’s NIS2 and the UK’s Cyber Security and Resilience Bill.

So the honest position is that a fit-for-purpose product is the front half of resilience, not the whole of it. You can build the safest cars in the world and still not be safe on the road, and that gap, the people, the survivability, the operation, is where resilience actually lives.

Accountability and Culture Are the Same Wall

I’ve long described cyber resilience as a hierarchy with leadership at the base, then culture, then governance, then defence, then collaboration. Last week I spent my time on the keystone, on the psychological safety that lets a team say “this feels wrong” before something ships. This week is the governance layer, and I want to rescue that word from the procedure manual it usually arrives in.

Governance, done right, is not bureaucracy. It’s the structure that makes leadership and culture load-bearing instead of decorative. Accountability without culture rots into theatre. Culture without accountability is goodwill, and risk homeostasis erodes goodwill every single time. You need the floor and the keystone, the rules that make the safe choice rational and the culture that makes people want to make it anyway. Either one alone is a wall with no wall behind it.

While I Was Writing This, the Floor Arrived

As I write this, in mid-June 2026, the argument has stopped being theoretical. Last week, the US government issued an export-control directive that forced Anthropic to disable two of its most capable models for every customer, after another firm claimed it could jailbreak one of them. Whatever you make of the specifics, and Anthropic itself disputes them and says it’s working to restore access, this is the floor arriving the hard way. Not as a considered building code that makes safety visible and owned, but as a sledgehammer swung overnight, on someone else’s timetable. It is exactly the failure mode I described a few paragraphs ago. When an industry does not build accountability deliberately, accountability eventually arrives anyway, blunt and from outside. The window I keep talking about did not close gently. Someone slammed it, see Joe Sullivan‘s latest piece.

Now I’m not arguing the directive was right or wrong, and the facts are still moving as I write. The lesson sits underneath that argument. There’s a thoughtful version of accountability, the kind that makes the invisible visible and assigns an owner, and there’s the version that lands as a shock because nobody built the thoughtful one in time. We just watched which one fills the vacuum by default.

And notice the other thing that directive exposed. It reached foreign customers and foreign nationals specifically, which means a capability that organisations elsewhere had built on disappeared overnight, not because it failed, but because of where it was governed from. That is a different risk from the ones I have been describing here, and it deserves its own piece. For now it is enough to say this: when critical capability concentrates under one jurisdiction, that jurisdiction’s politics quietly become part of your threat model.

To End

This is the arc of the three pieces, and I can see it clearly now I’m standing at the end of it. First, change the conditions around the problem, build for resilience and make compromise matter less. Then, when AI hands you a windfall, spend it on safety rather than letting it all flow into the accelerator, with culture as the keystone. And now, the part I’d been circling without naming – stop asking people to be heroes, and build the road so that the safe choice is the obvious, rewarded, default one.

We gave software its anti-lock brakes. What it still lacks is its crash-test rating, its liability regime and its MOT, the system that turned a generation of “please drive carefully” into cars that are simply safer whether the driver is a saint or not. The dividend is real. Whether it ever reaches safety will not be decided by goodwill. It will be decided by whether we make security visible, and someone accountable, while that choice is still ours to make rather than someone else’s to impose.

Next week I’ll be pulling that last thread – what cyber resilience even means when the off switch sits in someone else’s capital.

Now I Want to Hear from You

Of the four levers above, visible security ratings, vendor liability, a secure-by-design floor, and funded maintenance of the open-source commons, which one do you think would move the needle most over the next three years?

And here’s the harder half of the question…which of them is most likely to turn into compliance theatre if we get the design of it wrong, and what would stop that happening? Let me know over on LinkedIn, in the comments.