The impact question is really around scale; a few weeks ago Anthropic claimed 500 "high-severity" vulnerabilities discovered by Opus 4.6 (https://red.anthropic.com/2026/zero-days/). There's been some skepticism about whether they are truly high severity, but it's a much larger number than what BigSleep found (~20) and Aardvark hasn't released public numbers.
As someone who founded a company in the space (Semgrep), I really appreciated that the DARPA AIxCC competition required players using LLMs for vulnerability discovery to disclose $cost/vuln and the confusion matrix of false positives along with it. It's clear that LLMs are super valuable for vulnerability discovery, but without that information it's difficult to know which foundation model is really leading.
What we've found is that giving LLM security agents access to good tools (Semgrep, CodeQL, etc.) makes them significantly better esp. when it comes to false positives. We think the future is more "virtual security engineer" agents using tools with humans acting as the appsec manager. Would be very interested to hear from other people on HN who have been trying this approach!
To be honest this is an even bigger problem with Semgrep and other SAST tools. Developers just want the .1% of findings that actually lead to issues, but flagging patterns will always lead to huge false positive rates.
I do something similar as what you suggested and it does work well -pattern match + LLM. The downside is this only applies to SAST and so far nobody has found a way to address the findings that make up 90% of a security team's noise, namely SCA and container images.
I assumed I was still dealing with lots of false positives from Gemini due to using the free version and not being able to have it memorize the full code base. Either way combining those two tools makes the review process a lot more enjoyable.
100% agree - I spun out an internal tool I've been using to close the loop with website audits (more focus on website sec + perf + seo etc. rather than appsec) in agents and the results so far have been remarkable:
Human written rules with an agent step that dynamically updates config to squash false positives (with verification) and find issues while also allowing the llm to reason.
If I were to venture a guess, there's different world in which we might exist in the next 5-10 years.
In one of these futures, we, as auditors, seize to exist. If this is the future, then developers seize to exist too, and most people touching software seize to exist. My guess here is as good as any developer's guess on if their job will remain stable.
In another one of these futures, us auditors become more specialized, more niche, and bring the "human touch" needed or required. Serious companies will want to continue working with some humans, and delegating security to "someone". That someone could be embedded in the company, or they could be a SaaS+human-support system like zkao.
On the other hand, vibe coders will definitely use claude code security, maybe we should call it "vibe security"? I don't mean it as a diss, I vibe code myself, but it will most likely be as good as vibe coding in the sense that you might have to spend time understanding it, it might make a lot of mistakes, and it will be "good enough" for a lot of usecases.
I think that world is a bit more realistic today, than the AGI "all of our jobs are gone in the next years" doom claim. And as @zksecurityXYZ , I don't think we're too scared of that world.
These tools have been, and are making us stronger auditors. We're a small, highly specialized team, that's resilient and hard to replace. On the other hand large consultancies and especially consultancies that focus on low hanging fruits like web security and smart contracts are ngmi.
Auditors though, they are cooked.
I think you're massively underestimating the complexity and depth of a good security audit service.
Padme: You're scanning for vulnerabilities so you can fix them, Anakin?
Anakin: ...
Padme: You're scanning for vulnerabilities so you can FIX THEM, right, Annie?
but there are open versions available built on the cn OSS models:
I hope Anthropic can place alerts for their team to look for accounts with abnormal usage pre-emptively.
This is different than someones "npm audit" suggesting issues with packages in a build and updating to new revisions. Also different than iterating deeply on source code for a project (eg: nginx web server).
We have a different approach, in that we're using SAST as a fast first pass on the code (also helps ground the agent, more effective than just asking the model to "act like a security researcher"). Then, we're using pre-computer static analysis artifacts about the code (like data flow graphs, control flow graphs, dependency graphs, taint sources/sinks) as "data sources" accessible to the agent when the LLM review kicks in. As a result, we're seeing higher accuracy than others.
Haven't gotten access to this new feature yet, but when we do we'd update our benchmarks.
Fascinating! Our team has been blending static code analysis and AI for a while and think it's a clever approach for the security use case the Anthropic team's targeting here.
If this thing actually does something interesting, they're doing their best to hide that fact behind a steaming curtain of bullshit.
Another question that came up in conversations with them: there might be value in offering a nonscalable, high-touch service, where you build and maintain customized agents tailored to a client’s specific codebase on a periodic basis.
I don't yet have access to Claude Code Security, but I think that line of reasoning misses the point. Maybe even the real benefit.
Just like architectural thinking is still important when developing software with AI, creative security assessments will probably always be a key component of security evaluation.
But you don't need highly paid security engineers to tell you that you forgot to sanitize input, or you're using a vulnerable component, or to identify any of the myriad issues we currently use "dumb" scanners for.
My hope is that tools like this can help automate away the "busywork" of security. We'll see how well it really works.
Security teams are expensive and deal with huge streams of data and events on the blue side: seems like human-in-the-loop AI systems are going to be much more effective, especially with the reasoning advances we've seen over the past year or so.
We have been developing our own system (1) for several years, and it's built by engineers, not Claude. Take a look — maybe it could be helpful for your case.
Enough of this frontier grifting. Make it testable for open source developers at no cost and without login or get lost. You won't of course, because you'd get an unfiltered evaluation instead of guerilla marketing via blog posts, secrecy, and name-dropping researchers that cannot be disclosed.
No one knows you also caused that problem.