Part 3 of 5 in Foresite’s From Prototype to Production series, exploring building multi-agent systems in Google Cloud SecOps. |
In Part 1 of this series, we showed why the most important metric in an Agentic SOC isn’t speed-it’s knowing when your AI says "I don't know." In Part 2, we walked through our dual-loop evaluation harness, where Metric-in-the-Loop guides human analysts and Human-in-the-Loop continuously recalibrates production models against live holdout datasets.
Today, that architecture allows our SOC to cut out more than 80% of incoming alert noise before it ever consumes an analyst’s queue.
When security leaders hear that number, they usually ask what type of agents and prompt template we use. The honest software engineering answer is that cutting 80% of alert noise safely has almost nothing to do with prompt engineering or the agents themselves. It comes down to three foundational systems that most AI teams either conflate or ignore entirely: how you score evidence (Verdicts), how you structure reasoning (Judgements), and how you mathematically bind the two together (Confidence).
Most teams building security agents today rely on what we call "vibe evaluation." They run an agent against a test set of alerts, check whether the final output label matches Malicious or Benign, and ship to production if the pass rate looks high. In software engineering terms, that is the equivalent of testing a distributed payment system by only checking whether the API returns HTTP 200.
If you want to build production-grade security agents or evaluate a vendor claiming to run one - you have to move beyond pass/fail outcome checks and engineer the underlying mechanics of scoring, judgement, and confidence.
The single biggest architectural flaw we see in security AI is treating "Is this activity bad?" and "Did the detection rule work?" as the same question.
To eliminate alert noise without blinding your SOC, your data contracts must strictly separate every disposition into two orthogonal axes:
Malicious | Suspicious | Benign | UnknownTrue Positive | Benign Positive | False Positive | InconclusiveWhy is this distinction the key to cutting >80% of SOC noise? Because in a modern enterprise, the vast majority of alert volume is not caused by broken detection rules firing on phantom data (False Positives). It is caused by working detection rules firing on real, authorized administrative and business activity (Benign Positives).
Consider a classic example: a signed endpoint management tool runs a user-discovery command during a routine inventory check, triggering an EDR rule built to catch Living-off-the-Land (LOLBAS) reconnaissance.
Benign: It is authorized vendor software doing its job.Benign Positive: The detection rule did not misfire. The discovery command really ran, and the sensor caught the exact behavior it was engineered to detect.If your AI agent conflates Verdict and Judgement and labels this alert a False Positive, your detection engineering team receives a signal that the rule is noisy and broken. If they tune out that command to stop the "false positives," you just blinded your SOC to a technique real adversaries use after initial access.
By separating Verdict from Judgement, our system can recognize that the detection rule is healthy (Benign Positive) while proving that the specific execution is authorized (Benign)-allowing us to safely suppress the noise while keeping the sensor sharp.
Fig 1 — The verdict is Benign on both paths. Only the judgement decides
whether the detection survives.
When an AI agent arrives at a wrong verdict, teams often blame "LLM hallucination." In our experience, the AI agent is rarely the root cause. Almost always, an enrichment tool returned real telemetry that lacked structural context, and a naive scoring method misinterpreted what that data meant.
Rather than letting a language model eyeball raw JSON and guess a score, production systems should use a Spec-Driven Scoring Layer-a deterministic Weight-of-Evidence foundation that normalizes entities and weighs signals before an AI agent ever synthesizes a narrative. Because this layer is deterministic, we can replay thousands of historical alerts in seconds and audit how every piece of evidence contributed to the score.
All three traps below are Stage 1 failures. They happen at the entity level, before any alert-wide synthesis takes place. That separation is the practical reason we split per-entity investigation from alert-level synthesis in the first place: a scoring error on one entity is cheap to catch there and expensive to unpick once it has been folded into a verdict.
Here are three classic scoring traps that cause wrong verdicts in production, and the software engineering patterns required to catch them:
Imagine an employee at a major enterprise triggers an impossible-travel login alert. Your agent queries threat intelligence feeds for the corporate domain. Because that enterprise was the high-profile victim of a widely publicized breach two years ago, its corporate domain appears in dozens of threat intelligence reports alongside tags like adversary and compromise.
A naive scoring method sees high-severity threat intel hits and scores the domain as Malicious. Our evaluation harness caught this pattern early by inspecting entity-level score contributions: the system was confusing historical breach reporting with adversary infrastructure.
| The Engineering Fix: Scoring specifications must enforce a context priority cascade. Organizational boundary checks (verifying whether an asset or domain belongs to the customer’s own environment) must execute before generic threat-feed co-occurrence is allowed to contribute threat weight. |
Security analysts naturally look at history: "Has this indicator appeared in confirmed incidents before?" For an external IP address or a file hash, repeated appearance in past true-positive incidents is a strong signal of malice.
For a human identity, however, that logic breaks down. When we audited why certain legitimate employees were receiving elevated threat scores, we found they had appeared in multiple prior confirmed incidents-not as threat actors, but as the targets of recurring phishing campaigns.
The Engineering Fix: A sound scoring method requires Entity-Role Separation. Historical co-appearance must be gated by entity type and role: an external indicator can accumulate threat reputation across cases, whereas historical appearance on an internal user identity should never independently push a verdict beyond Suspicious without fresh behavioral evidence. |
Security telemetry is notoriously inconsistent. A single endpoint alert might report the exact same command execution three times in the same payload-once with a full system path, once as a bare binary name, and once in uppercase.
If your scoring engine treats each string as a separate artifact, the two bare strings (which lack the file-path context proving the binary came from a trusted vendor directory) will outvote the one fully qualified string 2-to-1.
| The Engineering Fix: Canonical Entity Deduplication must happen at the ingestion boundary. Before any scoring rule evaluates an alert, all surface variations of an indicator must be collapsed into a single canonical entity so the richest context wins. |
Fig 2 — In all three, the model was fine. The scoring layer read real
telemetry without the structure to interpret it.
Catching a wrong verdict is relatively easy once you compare the final output against human ground truth. Catching a wrong judgement-especially when the agent arrives at the right verdict for the wrong reason-is where traditional pass/fail evaluations completely fail.
To prevent what AI engineers call "context rot" (where a massive, monolithic prompt loses focus as alert data grows), we break reasoning down into modular Domain Skills-specialized analytical workflows for email security, command-line behavior, authentication patterns, and cloud infrastructure that load dynamically only when an alert’s shape requires them.
To evaluate how these skills form judgements, you cannot just grade the final output; you have to run Trajectory Evaluations that audit the intermediate reasoning path. Here are three judgement failures that trajectory evaluations catch:
During a regression run on a batch of benign alerts, our system returned the right Benign verdict 100% of the time. A standard pass/fail evaluation would have given the release a green light.
However, our trajectory monitor flagged an anomaly: internal consensus among the domain skills had dropped. When we inspected the intermediate judgement ledger, we found a subtle bug in a pattern-matching condition. Unclassified alert shapes were falling into a default "inconclusive" bucket that a legacy rule was translating into a small, automatic Malicious vote on every entity.
Because strong benign signals from other tools were outweighing that bug, the final verdict still came out Benign. Trajectory evaluation caught a silent reasoning defect that was artificially inflating threat scores and waiting to tip a borderline alert into a false escalation.
Consider an alert where a specialized email security sensor detects that an employee clicked a malicious phishing link. An email-focused agent skill checks the sender domain, sees that the vendor domain has exchanged legitimate emails with the company for months, and judges the alert as a False Positive.
That is a catastrophic judgement failure-compromised vendor accounts are one of the most common vectors for business email compromise. Familiar history is context, not exoneration.
The Engineering Fix: Every domain skill must be governed by a strict Top-Down Evidence Cascade:
|
During an investigation, an analytical skill will often test a hypothesis in order to rule it out-for example: "Investigated potential DNS data exfiltration; payload volume is negligible and traffic is strictly internal to Active Directory, ruling out tunneling."
In many agent architectures, the aggregation layer sees a finding generated under the "Data Exfiltration" category and naively counts it as evidence supporting a threat. Trajectory evaluation enforces explicit directional polarity on every intermediate judgement-ensuring that a finding written to refute a risk mathematically reduces threat weight rather than adding to it.
Even when you have a disciplined scoring method for Verdicts and modular domain skills for Judgements, you still need a mechanism to govern how they interact. What happens when the score leans one way, the domain judgements are split, or half your enrichment tools timed out?
That governing mechanism is Confidence.
Early in our engineering journey, we ran a simple experiment to see if an AI agent should have any say in its own confidence score. We took a single production alert and ran it through the pipeline seven consecutive times.
That experiment proved a foundational rule of our architecture: AI agents are great at synthesizing narratives, but they must never be allowed to grade their own certainty.
Instead, we engineer confidence as a deterministic, structural calculation that acts as the connective tissue between Verdict and Judgement. At a high level, it balances three observable engineering ratios:
Malicious while the endpoint context points to Benign, consensus alignment drops-automatically compressing overall confidence and triggering a specialized disagreement-resolution workflow.user, IP, host, file hash, domain) and three enrichment queries time out due to an API outage, completeness drops to 40%.This is the recursive count-ratio identity we pointed at in Part 2. Every term is a ratio of counts taken over the actual payload rather than a tuned constant: how many judgements were deep rather than shallow, what fraction of them agree with the leading verdict, how many of the alert's entities came back enriched. The same identity runs at every scale we evaluate, from a single entity up to the full alert, which is why a change at one layer can be replayed and audited across all of them.
Because overall certainty is bound by all three factors -
Evidence Strength x Consensus Alignment x Telemetry Completeness
- look at how confidence keeps the entire SOC safe:
Malicious, but the Judgements disagree: Consensus drops → Confidence compresses → the system is mathematically prevented from paging an on-call engineer with a false high-severity escalation, routing it instead for targeted human review.Benign Positive, but enrichment tools timed out: Completeness drops → Confidence compresses → the system is mathematically prevented from auto-closing an alert while flying blind.
Fig 3 — Three counts taken over the payload. Not one of them a tuned constant.
For CISOs and security engineering leaders, the lesson is clear: you cannot cut 80% of your SOC’s alert noise if you don't trust how your AI grades itself-and you cannot build that trust with pass/fail "vibe evaluations." An agent that gets the right answer for the wrong reason in a demo will eventually fail silently during a real incident, and an agent that confuses a Benign Positive with a False Positive will quietly erode your detection coverage.
Whether you are building internal security agents or evaluating an MDR / Agentic SOC provider, ask three architectural questions:
Benign Positive) without telling your detection engineers that their working sensors are broken (False Positive)?
In Part 4, we’ll dive into Multi-Agent Orchestration and Disagreement Resolution: how we design specialized domain skills that avoid context rot, and what happens inside the architecture when two domain specialists look at the same alert and reach opposite conclusions.
Missed the earlier installments? Read Part 1: The Metric Nobody Publishes: When Your AI Says "I Don't Know" and Part 2: Metric-in-the-Loop Without Giving Up Human-in-the-Loop.
What is a trajectory evaluation in AI security? A trajectory evaluation audits the intermediate reasoning path an agent took, not just the label it produced. Foresite runs these on the Catalyst Triage Agent because a pass/fail check on the final verdict cannot detect an agent that reached the right answer through faulty reasoning, which is the failure that surfaces later during a real incident.
What does "earned clean" mean in alert triage? It means a benign verdict requires affirmative evidence explaining why the detector fired, rather than the absence of further alerts. Verifying that a port scan came from an authorized vulnerability scanner is earned clean. No secondary alerts firing is not. Foresite's scoring layer treats absence of evidence as zero information, not as evidence of safety.
Why does an AI agent sometimes reach the right verdict for the wrong reason? Because a strong signal from one source can mask a defect in another. In one Foresite regression run the Catalyst Triage Agent returned the correct benign verdict every time while a faulty rule was quietly adding a malicious vote to every entity. The verdict was right; the reasoning was broken and would have tipped a borderline case.
What is context rot in a security agent? Context rot is the degradation that happens when a single large prompt carries too much alert data and loses focus on the part that matters. Foresite avoids it by breaking reasoning into modular domain skills for email, command line, authentication and cloud, loaded only when an alert's shape requires them.
Can an AI agent suppress alert noise without weakening detection coverage? Only if it separates two questions that look like one: whether the activity was malicious, and whether the detection rule worked. An authorized admin tool triggering a rule is a benign verdict and a benign positive judgement. Labeling it a false positive tells detection engineers their working rule is broken, and tuning it out removes real coverage.