The Metric Nobody Publishes: When AI Says I Don't Know

Recently, Chris Martin (@thatsiemguy) published an exceptional, must-read blueprint for the security community: "Bringing Your Own Agents into SecOps SOAR Playbooks with Google ADK".

In his article, Chris demystified what it actually means to build an agent - drawing a clean line between simple prompt wrappers and genuine autonomous loops - and demonstrated how Google’s Agent Development Kit (ADK), Model Context Protocol (MCP) tooling, and SecOps SOAR playbooks come together. Near the end of his post, Chris dropped a truth bomb that every security practitioner needs to hear:

"The moment you cross the autonomy threshold, you trade a human-in-the-loop for a metric-in-the-loop. Evals are that metric. Without evals you haven’t crossed the production threshold, you just have a demo that worked the day you showed it."

We couldn't agree more.

At Foresite, we have spent the past year running an ADK-native, multi-agent triage, assess and verdict generation (TAV) engine in live production on Google Cloud Security. Our system processes high-volume, multi-tenant alerts across diverse EDR, SIEM, and cloud environments 24/7. When you take agents out of the lab and into a real Security Operations Center (SOC), you quickly discover that the hardest engineering problem isn't getting an LLM to produce a clever investigation when everything goes right. The hardest problem is what your system does when things are ambiguous, incomplete, noisy, or failing.

Welcome to Part 1 of our 5-part series: "Building Effective Multi-Agentic Security Workflows in Google Cloud Security: Software and AI Engineering Best Practices." Today, we’re tackling the single most critical, least-discussed metric in AI security: When and how your AI says "I don't know."

The AI Triage Paradox: Overconfidence Kills SOC Trust

If you browse the marketing brochures of modern "autonomous SOC" tools, you’ll see impressive claims: 99% accuracy, sub-minute resolution, and total autonomous closure. Yet, talk to any Tier 1, 2, or 3 SOC analyst who has triaged alerts alongside a black-box LLM, and you will hear a different story:

1. The Hallucinated Clean Bill of Health: An alert with missing telemetry lands as a confident BENIGN because "no malicious IOCs were identified": a classic absence-of-evidence fallacy that masks real intrusion activity.
2. The "Confidence: 99%" Mirage: A single ambiguous PowerShell string causes an LLM to panic and issue a MALICIOUS verdict with near-certainty, even though the script was legitimate administrative maintenance.
3. The Silent Failure: An API rate limit (HTTP 429), a context-window truncation, or a malformed JSON payload causes the agent to crash or fail silently, dropping the alert into an unmonitored void. In a high-stakes SOC, a false sense of certainty is far more dangerous than admitted uncertainty. An analyst can quickly review an alert marked UNKNOWN with a clear explanation of why the system couldn't decide. But an analyst who is burned by a hallucinated BENIGN or an uncalibrated MALICIOUS verdict will immediately lose trust in the entire agentic system and turn it off.

Meeting Every Stakeholder Where They Are

To make an autonomous agent work in production, you cannot design solely for the developer writing prompts. An Agentic SOC must meet four distinct stakeholders where they actually operate:

1. For the SOC Analyst: Context and Continuity, Not Dictation

Analysts don’t want an AI that acts as an infallible judge; they want a tireless junior partner that surfaces verified facts, extracts signal from noise, and highlights gaps.

• Calibrated Abstention: When our agent encounters missing data or conflicting evidence, it emits UNKNOWN or SUSPICIOUS, alongside actionable continuity notes: "Skipped endpoint deep-dive due to telemetry gap; Stage 2 / human analyst should verify peer host logs and MITRE lateral movement signals."
• Plain-English Synthesis: The agent provides structured summaries (What Happened, Why This Verdict, What to Watch For, Recommended Actions) paired directly with the deterministic mathematical weights that produced them.

2. For the SecOps Engineer & Playbook Author: Predictable, Contract-Driven Integration

SOAR playbooks require rigid reliability. If an agent endpoint errors out or returns unexpected schema variants, playbooks break.

• Two-Stage Decoupling: We split our architecture into two independent Google Cloud Run services powered by Google ADK:
  • Stage 1 : Deep entity-level investigation (parsing raw telemetry, classifying entities, establishing baseline signals).
  • Stage 2 : Cross-alert synthesis (parallel specialist panels for Identity, MITRE chains, Historical patterns, Social engineering, and final consensus aggregation).
• Strict Wire Contracts: Communication between SOAR and ADK services is governed by typed Pydantic models with zero hidden schema mutations. If a field is declared in the contract, both sides validate it explicitly.

3. For the AI / Platform Engineer: Zero-Drop Resilience & Telemetry

In production, LLMs experience latency spikes, provider outages, and token quota exhaustion.

• Parse-Level Gradients: Instead of treating JSON extraction as binary (success or crash), we evaluate outputs on a 5-tier gradient: strict → repaired → partial → salvaged → empty
• Quota & 429 Circuit Breakers: If Vertex AI hits rate limits (RESOURCE_EXHAUSTED), our agent doesn't drop the alert. It gracefully sheds load to deterministic heuristics, recording the state in a structured degraded_at: [quota_429] telemetry tag.
• Root-Cause Telemetry: Every single LLM invocation captures stop_reason, native token consumption, wall-clock latency, and thinking budgets. We never guess why an agent hesitated; the telemetry tells us.

4. For the CISO & Compliance Leader: Mathematical Decomposability & Two-Axis Accountability

A security leader cannot defend a black-box hallucination to a board or regulatory body.

• Two-Axis Disposition: We separate the technical Verdict (MALICIOUS, SUSPICIOUS, BENIGN, UNKNOWN) from the operational Judgement (TRUE_POSITIVE, BENIGN_POSITIVE, FALSE_POSITIVE, INCONCLUSIVE). A legitimate administrative script triggering an alert is accurately classified as BENIGN + BENIGN_POSITIVE, rather than being dismissed as a broken rule.
• Data-Driven Count Ratios: Every score, weight, and confidence interval is derived through deterministic count-ratios and Weight of Evidence (WoE) models over the actual payload-never hardcoded magic numbers or hidden prompt biases.

Benign plus benign-positive is a rule doing its job. Filed as a false positive, the same case becomes a tuning ticket that removes a working detection.

The Architecture at a Glance

How does this look in practice when deployed with Google ADK in Google Cloud?

The models produce observations. The deterministic engine produces the score. A named practitioner authorises anything irreversible.

1. Stage 1: Ingests raw alerts from Google SecOps SOAR. It parses evidence dynamically, executes entity-level analysis, and applies strict "earned-clean" checks. If an entity lacks affirmative benign proof, it refuses to mark it clean and passes an explicit abstention state forward.
2. Stage 2: Receives the preliminary envelope and spins up specialized ADK sub-agents in parallel—each evaluating a distinct cognitive dimension (e.g., identity compromise, MITRE technique progression, BEC/social engineering lures).
3. Deterministic WoE Aggregator: Reconciles the panel's findings using a monotonic Weight-of-Evidence mathematical model. The LLMs provide structured domain observations; the deterministic engine determines the math.
4. Composer & Telemetry: Translates the verdict into human-readable executive narratives while packaging full telemetry back into the Google SecOps Case Wall.

The Metric That Matters: Measuring What You Don't Know

When evaluating an Agentic SOC in production, the primary metric isn't how fast the agent says "Malicious" on known malware samples. The primary metric is its abstention fidelity:

First verdicts, July 2026. n = 3,724 agent first verdicts, not the 5,445 incident population: coverage reached 94% only in week four.

When an AI agent is calibrated:

• True Positives are handled with rich contextual receipts.
• True Negatives are earned through verified, affirmative evidence—not passive silence.
• Novel or degraded cases are surfaced immediately with full transparency, preserving SOC bandwidth for genuine investigations while preventing silent misses.

Join the Conversation

What this means for security leadership: Adopting an agentic SOC isn't about gambling on black-box autonomy-it is about establishing a defensible, mathematically auditable operational multiplier that safely scales analyst capacity without introducing silent blind spots.

Are you experimenting with Google ADK, building custom SOAR integrations, or wrestling with autonomous agent reliability in your SOC? Let me know your comments. Let’s connect on LinkedIn.

Stay tuned for Part 2, where we break down the two-stage pipeline architecture and the mathematics of count-ratio scoring!

FAQ

What is a good abstention rate for an AI triage agent in a SOC?

Foresite's Catalyst Triage Agent returned an explicit "unknown" verdict on about 12% of first verdicts in July 2026, 439 of 3,724, each escalated to a named analyst. Few vendors publish this figure, so no industry benchmark exists. The useful test is whether a vendor measures it at all.

Is a high "unknown" rate a sign of a weak AI agent?

No. It is an escalation rate, and those cases are still resolved by people. In Foresite's July 2026 census the Catalyst Triage Agent returned unknown on 439 of 3,724 first verdicts, alongside 58% benign, 26% suspicious and 4% malicious. A system that never returns unknown is answering questions it cannot answer.

Should an AI agent be allowed to close security alerts on its own?

Investigation and reasoning can run autonomously; irreversible or high-impact actions should not. Foresite grants autonomy where actions are reversible and read-only, and requires a named practitioner to authorise anything consequential before it executes. Where that line sits is a design decision, not a limitation of the technology.

What is the difference between a false positive and a benign positive?

A false positive means the detection rule fired on something it was never built to catch. A benign positive means the rule correctly caught the activity it was designed to catch, and that activity turned out to be legitimate. Foresite records the two separately so working detections do not get tuned away.

Can Google's Agent Development Kit run inside a SecOps SOAR playbook?

Yes. The ADK Runner can be invoked from a Python action inside a Google SecOps SOAR playbook, so an autonomous agent executes alongside deterministic automation steps. Chris Martin of Google Cloud documented the pattern in detail, as a community prototype rather than an official Google Cloud release.

What should I ask an agentic SOC vendor before buying?

Three questions. What is the system allowed to say when it does not know, and at what rate? Where does the autonomy boundary sit, and who drew it? How would a specific automated decision be reconstructed and defended months later? A vendor who cannot answer all three on a call is describing a roadmap.