Agentic Vulnerability Research for Code Assess

securitycoding-agentsopen-sourceagentic-securityvulnerability-discoverysecure-code-reviewsastmulti-agent-orchestrationappsec-automationpatch-validation

Overview

Agentic vulnerability research for code covers tools that orchestrate LLM agents across staged pipelines to find, validate, and sometimes patch security defects in source code. The pattern is consistent across implementations: tool-mediated analysis broken into discovery, evidence gathering, skeptical validation of candidate findings, and reporting, with optional remediation and patch verification at the end. Google describes this as staged, expert-orchestrated agentic source code review, and the same decomposition shows up in the broader agentic vulnerability discovery literature.

What moved this onto the radar is that the category stopped being demos. open-kritt is a self-hosted, open-source orchestrator for finding and validating issues in code and has drawn meaningful stars since July 2026; Capital One's VulnHunter and Visa's VVAH are production tools from regulated institutions published as open code; OpenAI's Aardvark is now Codex Security, available as a research preview; and commercial platforms are folding agentic scanning alongside deterministic engines rather than shipping it standalone.

It sits in assess rather than trial because the evidence for equivalence with established controls is not there. An empirical study explicitly asks whether open-source LLM agents can replace SAST tools and evaluates them on precision, recall, and false positives — a question that is still open. Treat these tools as augmentation to secure code review and SAST, and design an evaluation, not a migration.

Adoption Signals

  • Open-source traction around open-kritt, a self-hosted orchestrator for agent-driven discovery and validation in code, which passed 2,000 stars since July 2026.
  • OpenAI's Aardvark graduated from private beta into Codex Security as a research preview, rolling out to ChatGPT Enterprise, Business, and Edu via Codex web.
  • Two Fortune-scale regulated enterprises published internal tooling: Capital One's VulnHunter shipped Apache 2.0 as composable Claude Code skills with a headless CI/CD agent and benchmarking harness after validation across thousands of repositories, and Visa's VVAH grew from 595 stars in July to over 2,300 by late August 2026.
  • Google's threat intelligence team disclosed the architecture of its Agentic Vulnerability Discovery Harness, used in proactive reviews, pentests, red team operations, and incident response, and positioned as a layer alongside continuous scanning.
  • Commercial consolidation: Cycode now routes deterministic and agentic analysis as one system, explicitly framing model choice, scan cadence, and auditability as the operative questions.
  • A supporting framework and benchmark ecosystem is forming, including CVE-Factory and LiveCVEBench with 190 tasks across 14 languages and 153 repositories, alongside agent frameworks such as Alias Robotics' CAI — whose archival status is itself a reminder that projects in this space churn.

Risks

  • Not a SAST replacement. Independent evaluation of open-source LLM agents against static application security testing tools treats substitution as an open empirical question, so retiring rule-based controls on the strength of agent findings is premature.
  • Write access by default. VVAH edits source files unless operators explicitly restrict it to detection-only mode, and it runs adversarial validation before any human sees the fix.
  • Validation and audit trail are the hard part. Commentary on VVAH argues the durable value sits in the validation layer and audit trail — proving to an auditor that a machine-written patch was reviewed by something — and Cycode frames "which model produced which finding" as a question most AppSec programs were never designed to answer.
  • The harness expands your attack surface. The Cloud Security Alliance logged more than 30 MCP-ecosystem CVEs between January and March 2026, including a CVSS 9.6 pre-auth RCE path, and documented sub-four-hour weaponization of an agentic framework auth flaw.
  • Prompt injection reaches the analysis pipeline. A systematization of prompt injection against agentic coding assistants synthesizes 78 studies on attacks through skills, tools, and protocol ecosystems — directly relevant when the agent is pointed at untrusted repositories.
  • Vendor and project volatility. CAI went from de facto open framework to a single archival commit inside eighteen months; treat any single tool in this category as replaceable.

Pros & Cons

Advantages

  • Multi-agent harnesses stage the work into discovery, skeptical validation, and reporting, which Google's AVDH team credits for a measurable leap in efficacy over unstructured model prompting.
  • Agentic analysis can reason about semantic flaw classes that rule engines cannot express, as Cycode reports for authorization CVEs its deterministic engines missed in benchmark testing.
  • Credible open-source implementations now exist to evaluate without vendor commitment, including open-kritt, Capital One's VulnHunter, and Visa's VVAH remediation harness.

Disadvantages

  • Empirical work questions whether open-source LLM agents can stand in for static application security testing tools, so replacing existing SAST controls is not yet defensible.
  • Remediation-capable harnesses such as VVAH edit source files by default and require operators to explicitly opt into detection-only mode, which is an easy misconfiguration with production consequences.
  • The agentic tooling layer is itself an attack surface: the Cloud Security Alliance recorded more than 30 MCP-ecosystem CVEs in sixty days, and one PraisonAI auth flaw was weaponized in under four hours.

Recommendation

Stand up one agentic harness against repositories you already know well and measure it, rather than pointing it at your crown jewels first. Run open-kritt or VulnHunter in parallel with your existing SAST for a quarter and score findings on precision, recall, and false positive volume — the same axes used in the SAST comparison study — so the assess decision rests on your codebase, not a vendor benchmark. Public benchmarks such as LiveCVEBench are useful for triaging candidates but are not a substitute for a local baseline.

Keep agents in detection-only mode initially. If you evaluate a remediation harness, pin the configuration explicitly — VVAH writes to source files by default — and require human review plus an audit record for every machine-authored patch. Borrow the structural lesson from Google's AVDH: the efficacy gain came from enforced skeptical validation steps and injected human expertise, not from a stronger model. Budget for triage capacity accordingly; unvalidated agent output will consume AppSec attention rather than free it.

Treat the harness itself as production infrastructure under threat. Isolate it, scope its credentials and tool access narrowly, pin and monitor any MCP servers it consumes given the documented CVE volume in that ecosystem, and assume untrusted code under analysis may attempt prompt injection through skills and tools. Revisit next quarter: if validated-finding precision holds and audit trails satisfy your compliance reviewers, this is a trial candidate.

Sources