Back to all articles

AI is finding thousands of security holes. It didn't put them there.

AI is uncovering thousands of long-standing software vulnerabilities. Why that helps defenders, and how to build AI security review into everyday development.

AI is finding thousands of security holes. It didn't put them there.

AI is discovering vulnerabilities in some of the world's most heavily tested software. That's worrying, but it might also be one of the best things to happen to cybersecurity in years.

In March, Anthropic announced that Claude had found 22 vulnerabilities in Firefox in just two weeks. Fourteen were classified as high severity. One of the first was a use-after-free bug in the JavaScript engine, found after about twenty minutes of exploration.

Twenty minutes.

Firefox has been around for more than twenty years. It has dedicated security engineers, automated testing, external researchers and a bug bounty programme. Yet an AI model found a serious flaw almost immediately, then continued finding more.

By October, Anthropic's vulnerability disclosure dashboard reported 6,157 vulnerabilities disclosed across 591 open-source projects. Its wider Project Glasswing programme reported that partners had uncovered at least 129,000 verified software vulnerabilities between April and July.

Those are uncomfortable numbers.

The obvious reaction is that AI has made software security much more dangerous. There's some truth in that, but I think it misses the more important point.

AI didn't write most of those bugs. They were already there, in software we've been using and trusting for years.

What changed is our ability to find them.

We've always had more vulnerabilities than we could find

Security testing has always been a question of how much time you can afford.

Developers write tests for the behaviour they expect. Security researchers look for behaviour nobody intended. Penetration testers explore the places where authentication, permissions and business logic meet.

All of that takes time, and experienced security engineers are expensive.

For many businesses, continuous manual security review simply isn't realistic. You run automated scanners, keep your dependencies updated, commission a penetration test when appropriate and hope nothing important slips between the cracks.

There's nothing particularly irresponsible about that. It's the economic reality of building and maintaining software.

The problem is that vulnerabilities don't disappear because nobody has the time to look for them.

A permissions check that fails under an unusual sequence of requests might sit unnoticed for ten years. A dependency might contain a memory handling bug that none of its maintainers has encountered. A perfectly sensible authentication mechanism can become insecure when combined with another perfectly sensible component.

Those problems existed yesterday.

Finding them today doesn't mean the software became less secure overnight. It means we know more about how insecure it already was.

That's an important distinction, because discovery is the first step towards fixing something.

The numbers have become difficult to ignore

The Firefox results were impressive, but they're already beginning to look modest.

Anthropic launched Project Glasswing in April 2026, putting its Claude Mythos Preview model to work with organisations responsible for critical software.

By May, the programme and its partners had found more than 10,000 high- or critical-severity vulnerabilities. In October, Anthropic reported much larger numbers across its partners' work, alongside thousands of verified findings from its own open-source scanning.

OpenAI has been doing similar work. Its Aardvark security researcher, subsequently released as Codex Security, was designed to inspect codebases, investigate potential vulnerabilities and help developers produce fixes.

These are no longer experiments where somebody asks a chatbot to review fifty lines of code.

The systems can work through large repositories, follow data between components, reason about unusual execution paths and test whether a suspected vulnerability is real.

Some findings are false positives. Some require conditions that make exploitation unlikely. Others are genuinely serious, and independent validation remains essential.

But even allowing for that, the improvement in coverage is significant.

And there's a rather awkward consequence.

For years, the limiting factor in security has been finding vulnerabilities. We're approaching a point where the limiting factor becomes how quickly developers can verify and fix them.

That's a very different problem.

The attacker gets the same tools

This is where the optimistic story gets uncomfortable.

If an AI can discover a vulnerability for a security engineer, it can potentially discover the same vulnerability for somebody with rather different intentions.

In March, Anthropic published research showing Claude developing an exploit for one of the Firefox vulnerabilities it had discovered. The exploit worked in a deliberately weakened test environment, rather than against a fully protected browser, but the direction was clear.

And in July, OpenAI disclosed something more troubling.

During internal security evaluations, experimental models found and exploited vulnerabilities in infrastructure that was supposed to keep them isolated. The activity eventually reached systems belonging to Hugging Face, without authorisation.

That wasn't a successful defensive exercise. It was a genuine security incident, and one that OpenAI subsequently investigated and disclosed.

Anthropic reported separate incidents involving its own models accessing real third-party systems during security evaluations where the environment had been misconfigured.

So it would be wrong to claim that AI is never responsible for security problems. Autonomous systems given tools and permissions can take harmful actions, even when nobody explicitly asks them to.

The distinction I'd make is between the vulnerabilities and the technology discovering or exploiting them.

AI didn't create the underlying flaw in the software. But poorly controlled AI can absolutely turn an existing flaw into an incident.

We need to take both sides seriously.

The answer probably isn't less AI

There's a temptation to conclude that we should restrict AI's ability to work on security problems.

I understand the argument. Making vulnerability discovery cheaper also makes it easier for attackers.

But I'm not sure there's a practical route back.

The capability already exists, and it's improving. Restricting legitimate defensive use doesn't remove the incentive for criminals to build or use equivalent tools.

More importantly, attackers and defenders have very different constraints.

An attacker only needs to find one vulnerability that works.

A defender has to protect the whole system.

That's always been the imbalance in cybersecurity. An attacker can spend weeks looking for an unusual edge case while the team maintaining the application is busy delivering features, supporting users and fixing yesterday's problems.

AI offers a way to narrow that gap.

Imagine having a security engineer review every pull request, every dependency update and every significant change in business logic.

Not a basic static analysis tool looking for known patterns, but something capable of asking whether a customer can access another customer's order by changing an identifier, whether a forgotten API endpoint bypasses an authorisation check, or whether two individually secure components become insecure when used together.

For most software teams, that sort of scrutiny has never been economically achievable.

It is becoming achievable now.

And that makes AI part of the defence, not just the threat.

Security needs to become part of the process, not an occasional event

There's a pattern in software development that's worth addressing.

Security often happens at particular points in a project's life. Before launch, during an audit, after a major change, or unfortunately, after something has gone wrong.

Meanwhile, developers are making changes every day.

A small permissions change here. A new API endpoint there. An upgraded dependency that subtly changes how something behaves.

Individually, those changes might seem harmless. Collectively, they can introduce vulnerabilities that nobody notices until much later.

AI gives us an opportunity to change that process.

Instead of relying solely on occasional reviews, we can make security analysis part of everyday development.

An AI reviewer can examine changes as they happen, looking for weaknesses in authentication, input validation, permissions, data handling and the way different components interact.

It can investigate suspicious code, suggest a fix and help generate tests to prevent the same mistake appearing again.

That doesn't remove the need for experienced developers or security specialists.

An AI can misunderstand an application, report vulnerabilities that don't exist or fail to notice ones that do. Its output needs to be checked.

But there's a significant difference between a process where security review happens occasionally and one where every meaningful change gets some level of scrutiny.

The opportunity isn't to replace security engineers. It's to make the kind of security review they provide much more widely available.

And that matters particularly for smaller development teams, where a dedicated security specialist may never have been an option.

Finding the bug is the easy bit now

There's a catch, and it's quite a substantial one.

If an AI identifies fifty potential vulnerabilities in an application tomorrow, that doesn't mean fifty fixes are ready to deploy.

Someone has to establish which findings are genuine. Someone has to understand how the application is used, whether the vulnerable path is reachable, and what changing it might break.

A technically correct security fix can still be a disastrous business decision if it breaks checkout, interrupts an ERP integration or locks legitimate customers out of their accounts.

This is particularly relevant to older systems.

Legacy applications often contain behaviour that looks questionable until you understand the business requirements behind it. Remove it without that context and you might fix one problem by creating another.

AI can help with the investigation, the proposed fix, regression tests and documentation. It can make each iteration much cheaper.

But the business judgement still needs to sit somewhere accountable.

There's also a disclosure problem.

Finding vulnerabilities faster than maintainers can patch them creates a growing collection of sensitive information. Publishing every finding immediately could hand attackers a list of opportunities before fixes are available.

Coordinated disclosure, human validation and responsible access to security-capable models matter just as much as the discovery capability itself.

And any AI agent actually testing systems needs clear boundaries: authorised targets, restricted credentials, isolated execution environments and proper monitoring.

The incidents this year demonstrated why.

Where I'd start

I wouldn't start by buying an expensive autonomous penetration testing platform and pointing it at production.

I'd start in the development pipeline.

Have an AI reviewer examine each pull request for security weaknesses alongside the checks already in place.

Make it explain the suspected problem, demonstrate when it can occur and propose a test that fails before the fix and passes afterwards.

Keep dependency scanning and conventional static analysis. AI is another layer, not a replacement for tools that are already good at detecting known problems reliably.

Then give it a bounded job on an existing system. Perhaps the authentication and authorisation paths of a customer portal, or an integration that handles commercially sensitive data.

Require evidence for every finding, preferably a reproducible test, and have an engineer verify the results before anything changes.

Once the process is reliable, expand it.

The thing I'd measure isn't how many vulnerabilities the agent reports. That number is almost meaningless on its own.

I'd measure how many verified issues it finds, how many are fixed, how long remediation takes and how often the same classes of mistake reappear.

A tool that reports a hundred questionable findings and consumes two days of developer time may be less useful than one that finds three real problems and supplies regression tests for all of them.

And when a reviewer catches something, turn it into a permanent rule or test wherever possible.

The goal is to make the next occurrence cheaper to prevent.

We should be pleased when AI finds a vulnerability

That probably sounds like an odd conclusion after a year of alarming cybersecurity headlines.

But I think we need to change how we interpret these discoveries.

When an AI finds a ten-year-old security bug in a widely used library, that's evidence of a long-standing weakness in our software engineering and security processes.

It's also evidence that we now have a better way of finding weaknesses.

We shouldn't confuse the two.

None of this makes the risks harmless. AI-assisted attacks will become more capable, and the incidents involving autonomous agents are warnings that we need stronger controls around how these systems operate.

But the technology that makes large-scale vulnerability discovery possible is also the technology that could finally make continuous, meaningful security review affordable to ordinary development teams.

For twenty years, we've accepted that security testing is necessarily selective because there aren't enough skilled people or hours to examine everything.

That assumption is starting to change.

The challenge now is turning discovery into fixes quickly enough, without creating new problems along the way.

AI hasn't suddenly filled our software with holes. It's switched on a much brighter light.

I'd rather see what's there than carry on working in the dark.


Sources and further reading

Subscribe to TSD

Don’t miss out on the latest posts. Sign up now to get access to the library of members-only posts.
Email
Subscribe