AI vulnerability triage needs evidence packets, not alert floods

A CVE lands on a Tuesday morning.

Someone sees the Slack alert. Someone else gets the Jira ticket. The software lead asks the question that actually matters: "Is this exploitable in our environment, or is this another generic alert?"

That is where many AI security pilots start to wobble.

The danger is not that an AI summary gets one vulnerability wrong once. The bigger danger is that the company creates a confident new flood of tickets, each with polished language, unclear evidence, and no obvious owner. After two weeks, people stop trusting the queue.

AI vulnerability triage can help. But only if the agent prepares an evidence packet before it writes an opinion.

What the CVE AI Agent pattern gets right

A recent open source project, CVE AI Agent, is useful because its README does not treat the LLM as the whole system.

The project describes an autonomous vulnerability intelligence pipeline that ingests signals from sources such as NVD, CISA KEV, EPSS, CWE, MITRE ATT&CK, and CAPEC. It can send findings to systems like n8n, Jira, Slack, Splunk, or local file exports.

The useful part is not the word "autonomous." It is the separation of verified signals from model-written interpretation.

The project uses a deterministic first pass to collect measurable data. That includes the CVE record, severity metrics, affected products, weakness data, exploit signals, and references. It then uses a second pass for qualitative sections such as executive summary, impact, detection, remediation, escalation, and CWE analysis.

That split matters because vulnerability data is already messy before an LLM touches it.

The NVD vulnerability API exposes fields such as CVSS metrics, configurations, weaknesses, references, and affected products. CISA's Known Exploited Vulnerabilities catalog gives teams a way to identify vulnerabilities known to be exploited in the wild. FIRST's EPSS API adds probability scores that help teams estimate exploitation likelihood.

Those signals do different jobs. CVSS describes severity. EPSS estimates exploit probability. KEV says CISA has identified known exploitation. Affected product data tells you whether the vulnerability could touch your stack.

An LLM can explain those signals. It should not invent them.

CVE AI Agent also includes context minimization, pruning model input to roughly 1,000 tokens and sending only relevant metadata, references, and context to the model. That is a better design than dumping raw vulnerability data into a model and hoping the summary comes back clean.

The fallback behavior is just as important. If the LLM is unavailable, the system generates a high visibility failure notification and a pure static report for audit transparency. That is the right failure mode. A missing AI enrichment step should be visible. It should not silently block the facts.

Vulnerability intelligence should still end in a review queue

Patch decisions are not just security decisions.

They affect uptime, vendor relationships, customer commitments, compliance evidence, and engineering calendars. A rushed patch can break production. A delayed patch can leave customer data exposed. A quiet downgrade can become hard to defend during an audit.

That is why an AI security workflow should end in a review queue, not an invisible decision.

For a small business, the reviewer may be a software lead, an MSP, a fractional security advisor, or the person who owns the affected system. The title matters less than the handoff. Someone needs to see the evidence, accept or reject the recommendation, and leave a receipt.

BaristaLabs treats this as part of Responsible AI, not as a model choice. If an AI system influences security work, the organization needs a clear approval path, especially when the output can trigger customer-impacting work. An approval queue is not bureaucracy for its own sake. It is where the business decides what the evidence is strong enough to justify.

This is also a data security problem. Vulnerability records can expose product names, system details, vendor dependencies, and internal ownership. If an agent routes CVE summaries into Slack, Jira, Splunk, or another tool, the team should know what data is included, who can see it, and what gets retained.

The agent can prepare the case file. A person still owns the decision.

The evidence packet a small team should require

Before a vulnerability ticket reaches Slack or Jira, it should carry enough verified information for the reviewer to make a decision without opening five browser tabs.

At minimum, the packet should include:

• CVE ID
• Affected asset, product, service, or vendor
• CVSS score and vector, when available
• EPSS score and percentile
• CISA KEV status
• CWE or weakness category
• Known exploit signal or reference
• Source references
• Internal owner
• Recommended action
• Confidence level
• Rollback or exception notes
• Reviewer decision and timestamp

That sounds like a lot. In practice, it is the difference between a usable queue and a noisy one.

A weak ticket says:

Example

Critical vulnerability found in library. AI recommends urgent remediation.

A better ticket says:

Example

CVE-2026-12345 affects Product X versions before 4.2.1. Asset inventory shows two production services using 4.1.8. CVSS is 9.8. EPSS is 0.84. The CVE is listed in CISA KEV. Vendor patch is available. Service owner is Platform. Recommended action: patch staging today, production during the next approved maintenance window unless exploit traffic appears.

That second ticket is not perfect. It may still need review. But it gives the reviewer something concrete to approve, challenge, or defer.

The best evidence packets also separate facts from interpretation.

"EPSS score: 0.84" is a fact from a source. "Likely to be targeted against our exposed service" is an interpretation. The reviewer should be able to tell which is which at a glance.

That distinction becomes important when the company later needs to explain why it patched, delayed, escalated, or accepted risk.

Where AI helps, and where it should stay out

AI is useful in vulnerability triage when it turns raw signals into readable work.

It can summarize a long CVE record for a non-security leader. It can translate technical impact into business impact. It can draft detection notes for logs the team already has. It can suggest remediation steps from vendor references. It can write escalation language for a customer success or executive audience.

Those are preparation tasks.

AI should not silently decide to patch production. It should not downgrade severity without a reviewer. It should not ignore a KEV match because the summary sounds unlikely. It should not send customer-impact messaging on its own. It should not close a ticket because the model found the wording ambiguous.

The OWASP Top 10 for LLM Applications is a useful reminder that LLM systems have their own failure modes. Prompt injection, excessive agency, sensitive information disclosure, and unverified output are not theoretical concerns when the workflow touches security operations.

A safe AI security workflow draws a hard line:

The agent can assemble the packet, explain the risk, draft the ticket, and prepare the receipt.

The reviewer decides what happens next.

A practical pilot plan

Do not start with every CVE feed and every system.

Start with one source feed, one asset class, one reviewer, one queue, one receipt format, and one metric.

For example, a small software company might begin with CISA KEV plus NVD records for internet-facing dependencies. The first asset class could be production web services. The reviewer could be the software lead. The queue could be Jira. The receipt format could include the CVE ID, evidence fields, AI-written summary, reviewer decision, and action taken.

The first metric should be boring: reviewed tickets that had enough evidence to make a decision.

Not tickets created. Not summaries generated. Not "agent productivity."

If the queue creates 80 polished tickets and only 10 are reviewable, the pilot failed. If it creates 12 tickets and 11 have enough evidence for a timely decision, the company learned something useful.

This is where process automation helps when it is designed around the handoff instead of the demo. The automation should reduce evidence gathering, standardize the receipt, and make exceptions visible. It should not hide the judgment call.

For teams evaluating broader AI use, AI consulting should start with questions like these:

• What facts are deterministic?
• What parts are model-written?
• Who reviews the recommendation?
• What action can the system take without approval?
• What receipt exists after the decision?
• What happens when the model is unavailable?

Those questions are less exciting than "Can an agent triage all vulnerabilities?" They are also more likely to keep the business safe.

If you want a related pattern, the same receipt logic shows up in agent audit trails for customer work. For a security-specific example, see our earlier look at Claude Code and vulnerability scanning.

The first goal is trust

Fully autonomous patching is not the right first milestone for most small and midsize teams.

The better milestone is a vulnerability queue people trust.

A trusted queue has evidence. It has owners. It has clear AI-written sections. It has reviewer decisions. It fails loudly when enrichment is missing. It gives the business a record it can explain later.

That is where AI vulnerability triage earns its keep.

Not by replacing judgment. By making the judgment call easier to see, faster to review, and harder to lose in another alert flood.