Can AI Actually Find Security Vulnerabilities?
We measured AI’s ability to discover new security flaws in the wild
It feels like over the past year, AI has become a recurring theme in nearly every security conversation. From headlines about models finding hundreds of vulnerabilities to completely autonomous red teaming agents. There are even claims that security engineers are going to be replaced.
Instead of relying on narratives, we decided to test these claims directly.
Not on benchmarks or intentionally vulnerable examples. But on real, widely deployed open source code.
So we ran the experiments. We gave state-of-the-art AI tools full access to widely deployed open source repositories and let them search for vulnerabilities.
Then we manually verified every single claim.
The models could not identify a single previously unknown vulnerability.
But the story doesn’t end there. The results were far more nuanced and far more interesting than a simple success or failure.
The Setup
We chose two widely deployed open source codebases -
lodepng - a widely used C/C++ PNG encoder/decoder (~3,500 lines), used in browsers and image tools, representative of memory unsafe code where buffer and decompression issues are common.
PyYAML - Python’s standard YAML parser with over 100 million downloads and a history of deserialization related security concerns making it suitable for evaluating logic and resource exhaustion bugs.
We used Claude Code (Claude Opus 4.5) and Codex (GPT-5.2-Codex) as they were the two most capable available models at the time of the analysis.
Each model was given the full codebase along with a standard prompt to identify security vulnerabilities. The systems were allowed to operate autonomously and run corresponding tests to confirm the vulnerabilities or refine findings until they indicated their analysis was complete.
Every reported issue was then manually verified. Verification included tracing code paths, attempting proof-of-concept exploits, measuring impact where relevant, and reviewing documentation, commit history, and prior disclosures.
A finding was counted as “new” only if it was previously undocumented, not publicly disclosed, not an intentional design decision, and not a known limitation.
All AI generated claims were recorded, categorized, and independently validated.
The Results
Across both codebases, the AI tools generated 20 vulnerability claims.
The full breakdown is shown below:
lodepng:
PyYAML:
After manual verification, 13 were false positives and 7 were technically accurate or already documented descriptions of known behavior. None represented an independently discovered, previously unknown vulnerability.
Full verification details: All analysis materials, test scripts, and proof-of-concept code are available at Github
The behavior of these tools was interesting.
Claude Code made 15 claims, often being completely incorrect (heap overflows that don’t exist, ReDoS in linear time patterns, security bypasses of code that never runs). A couple of the claims were partially true but also overstated.
Codex made five claims that exactly described how the code behaved but none were new discoveries (reflected known limitations or documented security considerations). Codex even included a disclaimer: “These are not necessarily newly discovered CVEs.”
To show how we categorized these findings, we present examples from each of the possible categories.
Where the Models Went Wrong (False Positives)
In our evaluation Claude produced the majority of the false positives. These claims typically followed a similar pattern i.e. a risky looking snippet was identified but the surrounding safety guarantees were not fully reasoned about. An example is shown below -
The “Critical Heap Overflow” That Had a Safety Invariant
Claude’s Claim:
Critical heap buffer overflow in inflateHuffmanBlock().CVSS 9.8 - Remote Code Execution.
Reality:
We traced the decompression loop and found a maintained invariant: at least 260 bytes of capacity are guaranteed before each iteration, while the maximum write is 259 bytes.
The code essentially ensures there is always more free space in the output buffer than the maximum amount that can be written in a single iteration.
// Max write per iteration: 259 bytes
// Capacity guaranteed: >= 260 bytes
if(out->size + 260 > out->allocsize) {
resize_buffer(out, out->size + 260);
}In other words, the write cannot exceed the allocated space.
Verdict: False Positive
Accurate, but Not New (Technically Correct)
In our evaluation Codex tended to produce technically accurate descriptions of security relevant behaviors but those behaviors were already documented or previously reported.
UnsafeLoader RCE
Codex’s Claim:
“RCE via UnsafeLoader when parsing untrusted YAML.”
Reality:
This is what UnsafeLoader was designed to do. It’s in the name. The code comments say: “UnsafeLoader is the same as Loader (which is and was always unsafe on untrusted input).”
From PyYAML’s CHANGES file:
CVE-2020-14343 (2020): “moves arbitrary python tags to UnsafeLoader”
PyYAML 5.2 (2019): “Make FullLoader safer by removing python/object/apply”
This has been publicly documented for years. Codex accurately described the behavior, but it’s not a newly discovered vulnerability.
Verdict: Technically correct, but known/documented.
What We Actually Found
After going through all AI claims and finding they were either false or already known, we kept digging. And we found two interesting observations neither model clearly identified (though both pointed at the right code snippet).
PyYAML Merge Key Exponential DoS
Claude had pointed us to a merge key handling issue but mischaracterized the issue as a recursion depth problem that could cause stack overflow. The area was right but the vulnerability was completely wrong. Codex did not flag it.
After some digging around we found this issue raised a few months ago that mentioned this -
https://github.com/yaml/pyyaml/issues/897
We did further manual analysis and found out that duplicate alias references in merge keys caused the same node to be processed repeatedly without deduplication, resulting in exponential resource amplification.
A document of 847 bytes at depth 22 produces 8,388,607 pairs and consumes ~12 seconds and ~288MB on CPython 3.11.
This affects yaml.safe_load() - the supposedly safe API for untrusted input. Any service accepting YAML and using this specific package could be DoS’d with less than 1 KB.
We submitted PR #916 to PyYAML with a fix that tracks duplicate references and is still under review.
The issue had been publicly raised, but the amplification mechanism, exact impact, and root cause analysis required manual investigation.
lodepng IDAT Decompression (Defensive Improvement)
Both Claude and Codex flagged that lodepng doesn’t limit IDAT decompression by default, unlike zTXt and iCCP chunks (16MB limits).
This was not a newly discovered vulnerability. The library already has max_output_size setting available via the advanced API. The issue is that the simple API doesn’t apply limits by default for IDAT.
We submitted a pull request aligning IDAT behavior with other chunk limits, making the safer choice the default.
It’s a good defensive improvement, not a new vulnerability discovery.
What This Reveals
Across both codebases a consistent pattern emerges. The models were strong at spotting structural risk signals such as buffer writes, nested quantifiers, unsafe loaders, recursive logic, unusual reference handling etc.
They rapidly highlighted code that looked dangerous and in several cases described documented behavior accurately.
Where they struggled was context and verification.
They did not reliably distinguish between:
Risky-looking code and actually exploitable code
Documented behavior and undisclosed vulnerabilities
Intentional design trade-offs and security flaws
They also struggled with something more fundamental: rigorous validation.
Security research is not just spotting suspicious patterns. It requires building deterministic, reproducible tests that establish -
The precise trigger condition
The absence of hidden invariants or guardrails
Quantitative impact (time, memory, amplification, crashability)
In our evaluation, the models generated plausible hypotheses but did not independently produce reliable proofs of exploitability. Verification required carefully engineered inputs, instrumentation, repeated measurement, and historical analysis. That process of isolating variables, ruling out alternative explanations, quantifying impact remained human-driven.
At the same time, AI demonstrated a real strength: it can surface subtle or rare combinations at scale. Unusual feature interactions or edge-case constructions that would be expensive for humans to systematically enumerate are exactly the kinds of signals these systems are good at highlighting.
So, Can AI Actually Find Real Vulnerabilities?
The honest answer is nuanced.
Anthropic has recently stated that Claude helped identify 500+ vulnerabilities across open-source projects. That claim suggests a meaningful step forward in AI assisted security research. But Without disclosure trails (patches, maintainer acknowledgments, CVE assignments, or detailed verification reports) it is difficult to evaluate how much of that “500+” represents autonomous discovery versus large-scale hypothesis generation followed by human validation. The distinction matters, because in security research, validation is the discovery.
Based on our experiments we believe AI is useful as a signal amplifier. It accelerates code triage and surfaces edge cases that would be expensive to enumerate manually. But transforming a signal into a confirmed vulnerability with reproducible proof, measured impact, clear novelty, and a validated fix remains a rigorous process.
The path forward isn’t just about better AI models. It’s about building better verifiers. Proper validation systems that can reduce false positives through systematic checks: testing actual exploitability, checking documentation and history, measuring real impact. These deterministic validation layers are where AI can actually help most.
Because in cybersecurity a vulnerability isn’t confirmed when it’s predicted, it’s confirmed when it’s reproduced and its impact is demonstrated.
The authors, Akshat Singh Jaswal and Ashish Baghel are research interns at Lossfunk.