When Your Agent Panel Agrees for the Wrong Reason

Why we replaced sequential pipelines with an evidence panel, and what we built after the panel agreed for the wrong reason

If you look under the hood of almost every multi-agent security product on the market today, you will find the same architectural blueprint: sequential task decomposition.

An alert fires. Agent 1 parses the entities. Agent 2 queries threat intelligence. Agent 3 checks user activity logs. Agent 4 reads the output of Agents 1, 2, and 3 and writes a summary. It looks clean on an architecture slide because it mirrors a traditional software pipeline.

In a live Security Operations Center, sequential task decomposition is fragile. Security investigations are not assembly lines; they are contested hypotheses. When you decompose an investigation into a daisy chain of narrow sub-tasks, an early framing error by Agent 1 quietly poisons the context window of Agent 2, which biases the query of Agent 3, and gets rubber-stamped by Agent 4. The pipeline compounds its first assumption. No stage in it exists to argue with the one before.

In Part 1, we explored why an agent must know how to say “I don’t know.” In Part 2 and Part 3, we covered how we evaluate and score those decisions using a continuous metric-in-the-loop harness.

In Part 4, we look at the core reasoning architecture itself: why we abandoned sequential task decomposition entirely, how we combined Andrej Karpathy’s LLM Council with scientific Panel-Based Weight of Evidence, and what happened when our AI situation room fell into a very human trap.

Fig 1 — A pipeline inherits its first framing.
A council has to earn its agreement.

1. Two Inspirations: The LLM Council Meets the Clinical Trial

Instead of chopping a security alert into assembly-line chores, we asked a different question: How do humans make high-stakes decisions when the data is noisy, incomplete, and contradictory?

Our architecture borrows deeply from two foundational paradigms outside traditional SOC automation.

A. Andrej Karpathy’s LLM Council, engineered for SOC economics

Andrej Karpathy recently popularized the concept of an LLM Council: instead of trusting a single model pass, you convene a panel of distinct expert personas to independently examine a problem, critique each other’s blind spots, and synthesize a collective verdict.

In a SOC, this is the equivalent of walking a complex alert into a situation room staffed by your best specialists:

• An Identity & Context Analyst who cares only about authentication baselines, privilege posture, and role fit.
• A Tactic-Chain Analyst who maps process trees and kill-chain progression.
• A Historical Pattern Analyst who looks at longitudinal alert cadence and prior analyst closures across the tenant.
• A Fast-Path & Disagreement Validator whose job is to challenge premature consensus.

Historically, running a multi-persona council on every alert was economically impossible. If you fire six frontier-model reasoning passes on every low-severity ping, you will bankrupt the SOC by Tuesday. We made the council economically viable on every single case by engineering deterministic shape-based activation and dynamic token budgeting. Specialists self-activate in parallel only when the mathematical shape of the evidence warrants their domain expertise, operating under a strict wall-clock budget that allocates deeper thinking tokens only where diagnostic lift is highest.

B. Panel-Based Weight of Evidence

Convening a panel of experts solves the perspective problem, but it introduces an aggregation problem. How do you combine their findings without resorting to a naive majority vote?

For that, we turned to Panel-Based Weight of Evidence, the structured scientific evaluation method used by regulatory bodies like the FDA and EFSA in toxicology and drug safety trials, as well as legal and administrative tribunals.

1. Assembling independent lines of evidence. The panel gathers all available data sources and groups them into distinct, independent lines. In toxicology, that means separating human epidemiological studies, in vivo animal trials, and in vitro cell assays. In our SOC, Stage 1 gathers independent telemetry lines: external threat intelligence, live behavioral process execution, identity context, and historical tenant closures.
2. Weighing individual studies. Not all evidence is created equal. In drug safety, an in vivo study demonstrating real-world biological outcomes carries far more weight than a preliminary in vitro computational assay. Our architecture enforces the exact same hierarchy: a deep, multi-signal behavioral investigation carries orders of magnitude more mathematical weight than a single static tag or heuristic match.
3. Panel ranking and minority signal protection. Experts score how strongly each line of evidence links to the central hypothesis. Crucially, minority or divergent data points are weighed against majority concordant findings rather than being automatically erased by them. If four in vitro assays look clean but one in vivo trial shows acute toxicity, the panel does not average them out to “mostly safe.”
4. Collective integration. The panel synthesizes the weighted categories into a quantitative and qualitative determination, whether that is a clinical hazard identification, a legal finding, or a SOC escalation verdict.

For twenty years, security automation meant brittle SOAR playbooks and if/else regex rules. Today, every alert that enters our SOC is evaluated with the structural and evidentiary rigor of an FDA clinical drug trial.

Fig 2 — Four evidence lines in, one posterior out.
Weighed by reliability, in vivo above in vitro, with minority signals protected.

2. The Situation-Room Rabbit Hole

When you build a system where weights of evidence are dynamically allocated and expert personas synthesize complex narratives, you unlock human-level analytical depth. You also unlock a classic human failure mode: situation-room groupthink.

Anyone who has worked in a real SOC has seen a room full of smart human analysts talk themselves down a rabbit hole. One analyst offers a plausible theory for why an anomalous alert might be harmless; a second finds historical context that supports the theory; a third notes that the user was not locked out; and within five minutes the entire room has constructed a brilliant, airtight explanation for why a real attack is nothing to worry about.

During production testing, we caught our AI panel doing the exact same thing.

The Microsoft Defender Graph case

Consider a real pattern we observed on alerts ingested via Microsoft Defender and Microsoft Graph telemetry.

Inside the alert payload was a hard indicator: a connection involving a genuinely malicious IP address flagged by the upstream detector. By right, that signal should have anchored an escalation. Yet when the panel convened, the verdict came back Benign.

When we inspected the panel’s reasoning trace, every step held up on its own. The logic read like a seasoned analyst team having a productive situation-room debate:

1. The panel recognized the malicious IP in the Microsoft Defender Graph logs.
2. The historical and contextual specialists noted that this specific alert pattern had appeared previously in the customer’s environment.
3. The panel then reasoned, collectively and persuasively, that because this pattern had been seen before and the customer had active Microsoft Defender controls in place, the customer’s existing automated rules or security team must have already detected and remediated the underlying issue.

The panel took a live malicious indicator and explained it away by assuming that historical familiarity meant someone else had already taken care of it. They agreed unanimously, for the completely wrong reason.

Fig 3 — Three quiet dimensions outvoted one smoking gun.
Nobody hallucinated.

3. How the Architecture Catches Panel Rabbit Holes

When an AI panel talks itself down a rabbit hole, you cannot fix it by tweaking a prompt to say “be more careful.” If the underlying aggregation math allows four contextual benign narratives to drown out one hard malicious indicator, the panel will eventually rationalize away another threat.

Over a series of recent architectural pushes led by our Applied AI engineering team, we overhauled both the epistemic rules of the council and the mathematical backstop that governs how panel votes are integrated. While our exact Weight-of-Evidence lookup tables and calibration formulas remain proprietary, the software engineering principles behind these safeguards apply to anyone building production multi-agent systems.

1. The Top-Down Evidence Cascade and the earned-clean mandate

The panel cleared the Microsoft Defender Graph case because it let contextual familiarity override present evidence.

Jonna Nevins, a Senior AI Engineer on my team, introduced a strict Top-Down Evidence Cascade across every specialist in the council. The cascade and the earned-clean standard we described in Part 3 both came out of this case. Agents are no longer allowed to weigh evidence in whatever order makes for the best story. They must evaluate claims in a strict, irreversible hierarchy.

Crucially, we banned speculative exoneration. A specialist is strictly prohibited from arguing that a threat is benign because “it occurred in the past without incident,” “an admin probably ran it,” or “the customer’s controls likely remediated it.” To cast a Benign vote against a fired detector, a specialist must produce an affirmative, earned-clean witness: a verified change ticket, a cryptographically signed administrative tool, or confirmed sandbox execution. Absence of secondary bad news is never proof of safety.

2. Targeted abstention: silencing uncorroborated clean votes

In a panel architecture, false negatives often happen because Stage 1 investigation retrieves deep evidence on one entity, the malicious IP, but finds nothing notable on three adjacent entities: the user’s recent logins, the host’s patch level, the time of day.

Previously, the specialists reviewing those quiet dimensions would see no bad news and cast confident Benign votes, inadvertently outvoting the single specialist holding the smoking gun.

3. Figure-ground residual protection

Borrowing from perceptual psychology and regulatory Weight-of-Evidence panels, our Scene Consolidator, the final deep-reasoning seat on the council, runs a figure-ground completeness check.

Attackers hide inside routine enterprise noise. An alert might contain 95% normal administrative background activity, the ground, and 5% malicious command-and-control traffic, the figure. Asking the panel “does this alert look mostly benign or mostly malicious?” invites the 95% benign context to drown out the 5% threat.

4. Replacing brittle override stacks with a unified posterior backstop

When software engineers discover that an AI system missed a threat, the reflex is to write a hard-coded safety floor: if detector == "Defender" and has_malicious_ip: never_auto_close().

Over six months you accumulate eight or nine of these well-intentioned one-way override rules stacked on top of each other. Your false-positive rate spikes because every override rule fires independently. When you relax one rule to give the SOC breathing room, false negatives immediately slip back through.

In one of our most impactful recent architectural reforms, Jonna tore out the entire stack of ad-hoc post-aggregation overrides, and simultaneously pruned a legacy shadow-calibration path, replacing them with a single Unified Posterior Disposition Layer.

Instead of brittle if/else patches arguing with the council, every specialist vote, Stage-1 behavioral threat finding, and detector claim feeds into a single, mathematically grounded Bayes minimum-cost Weight-of-Evidence posterior calibrated directly against our live production envelope.

Fig 4 — None of the four is a prompt instruction.
All four are architecture.

What This Means for Security Leaders

The industry is arriving at the same conclusion quickly: a sequential chain of prompts produces an echo chamber.

Convening a council structured around a Panel-Based Weight of Evidence gives us the best of both worlds, the multi-perspective analytical depth of an elite situation room paired with the evidentiary discipline of a clinical drug trial. And by engineering hard architectural guardrails — top-down evidence cascades, targeted abstentions on uncorroborated clean votes, figure-ground residual checks, and a unified Bayesian posterior — we ensure that when our panel agrees, it agrees on the evidence rather than on a convenient story.

Coming Up in Part 5

In Part 5, we close out the series by looking at how all of these pieces — abstention gates, continuous evaluation harnesses, scoring rubrics, and Weight-of-Evidence panels — come together in day-to-day SOC operations to transform the relationship between human analysts and autonomous agents.

Missed the earlier installments? Read Part 1: The Metric Nobody Publishes, Part 2: Metric-in-the-Loop Without Giving Up Human-in-the-Loop and Part 3: Why We Don’t Let the Model Score Itself.

FAQ

What is sequential task decomposition in a multi-agent system? Sequential task decomposition chains agents in a fixed pipeline, where each agent receives the output of the one before it and performs a narrow sub-task. It mirrors a traditional software pipeline, which makes it look clean on an architecture diagram. In a security investigation it is fragile, because an early framing error is inherited by every downstream agent and gets summarized as fact rather than challenged.

What is false consensus in a panel of AI agents? False consensus is a panel of agents reaching unanimous agreement through reasoning that is individually plausible and collectively wrong. It is not hallucination. Each specialist can be reporting accurately about the dimension it examined while the aggregation across them produces a verdict no single specialist would have defended alone, which is why output-level pass/fail testing cannot detect it.

Can AI agents outvote a confirmed threat indicator? Yes, if the aggregation math treats absence of evidence as evidence of safety. Foresite observed this in production: one specialist held a confirmed malicious IP while three others examined quiet adjacent dimensions, found nothing notable, and cast confident benign votes. Three uncorroborated clean votes outvoted one hard indicator. The fix is architectural rather than a prompt instruction.

What is a targeted abstention gate? A targeted abstention gate intercepts any benign vote that rests on the absence of threat intelligence rather than an affirmative clean witness, and forces that specialist to return unknown instead. Stripping uncorroborated clean votes of their voting power is what stops several quiet context checks diluting one real threat signal.

What is figure-ground residual protection in alert triage? Figure-ground residual protection subtracts every explained benign element from an alert and asks what remains unexplained, rather than asking whether the alert looks mostly benign. Attackers hide inside routine enterprise activity, so an alert that is 95% normal administrative background and 5% command-and-control will read as mostly benign to a proportional judgement. If any malicious indicator is still standing in the residual, a clean verdict is barred.