What the research actually says about AI-generated code security

Every article about this subject opens with a percentage, and the percentages disagree wildly. That is not because someone is lying. It is because "insecure" means five different things across five different studies, and almost nobody says which one they are using.

This page collects the figures that get quoted, names who produced each one and what it measured, and says where each is load-bearing and where it is not. We use several of these numbers ourselves, so being straight about their limits is self-interested as well as correct.

The headline numbers

Which of these mean what they appear to mean?

The Escape.tech numbers are the most useful and the least quoted. They scanned live production applications rather than benchmark tasks, which means the flaws counted are ones that survived somebody deciding the app was finished. A benchmark measures what a model produces; a production scan measures what people ship, and those differ because humans review some of it.

The GitGuardian comparison is the strongest single finding on this page. It is not a vendor scanning its own output; it is the same detection method applied to two populations of public commits, and the AI-assisted population leaks at roughly double the rate. A ratio between two comparable groups is far more robust than an absolute percentage, because whatever bias sits in the detector applies to both sides.

The 86% XSS figure needs its context restated. CSET prompted five models for tasks where XSS was reachable, so the denominator is "samples where the flaw was possible", not "all generated code". It is a real finding about how often models take the unsafe path when offered one, and it is not a claim that 86% of your app is vulnerable.

The spread from 38% to 62% is mostly definitional. Different scanners, different severity floors, different languages, different corpora. Anyone quoting one of them as the number is choosing a number, and the honest reading of all of them together is "a large minority of generated code carries something a scanner would flag".

What the research agrees on

Set aside the headline percentages and the studies converge on a shorter list. These are the findings that repeat across different methods and different teams, which is the only kind of agreement worth acting on.

Authorisation is the weakest area, consistently. Models produce authentication that works, because "let users log in" is a complete instruction, and produce authorisation that does not, because whether user A may read user B's record is a question the prompt never asked. This shows up as broken access control in every taxonomy that measures by category.

Secrets handling is measurably worse than the human baseline rather than merely as bad. That is the GitGuardian ratio, and the mechanism is not mysterious: a model asked to make something work will inline a value that makes it work.

Input handling is inconsistent rather than absent. Frameworks escape by default, models use the framework correctly most of the time, and then a request for formatted output takes them straight through the one escape hatch that turns the protection off.

Dependency selection is worse than people assume. Around one in five package names in generated code does not exist, which is only a build error until somebody registers the name.

What none of it tells you about your app

All of these are base rates across populations. Your app is one sample, and knowing that 44% of tasks in a benchmark introduced a flaw tells you nothing about whether yours did.

The failure modes are also specific in a way aggregate statistics erase. A study can tell you broken access control is common; it cannot tell you that in your app it is the /invoices/:id route, because you added tenant filtering to five handlers and the sixth was written on a different day. That is not a statistical property, it is a fact about your repository.

The practical move is to stop treating the percentages as a decision input. They are useful for deciding whether this category of risk deserves an hour of your week, and the answer to that is clearly yes. They are useless for deciding what to fix.

What to do with an hour instead

Read your own code, or have something read it. Two accounts and twenty minutes will tell you whether cross-account reads work in your app, which is worth more than every figure above combined, and the six checks we published elsewhere cover the rest of the common ground.

Past that, the limit is surface. A repository has more of it than anyone gets through in an evening, and the flaw that gets exploited is usually in the file nobody thought to open. That is what Sentrint does: it reads the whole repository, reports what an attacker would reach first in plain English, and gives you a fix you can paste back into whichever tool wrote the code. It is a code scan and not a pentest, so it reads what you wrote rather than attacking your running site, and the first scan is free with no card.

How this page is maintained

Numbers get added when a study is published with a stated methodology and a public write-up, and removed when the source turns out not to support the claim. If a figure here is wrong, or you have a study that belongs on it, the fastest way to reach us is the contact route on the site. A page of statistics is only worth linking to if somebody is prepared to take things off it.