Parallel Agents: Method, Contradiction, Reconcile

A practitioner's field guide to orchestrating many AI agents correctly — at the same time.

Preface

The promise of running multiple agents in parallel is speed. That is the wrong reason to do it. Speed is a side-effect. The real value comes from three mechanisms that only work when agents are genuinely independent and methodologically distinct. This book is about those three mechanisms, why they work, and how to build infrastructure that makes them reliable.

The first mechanism is method diversity. A single agent makes a single set of methodological choices — which layer to probe, which path to follow, which abstraction to trust. Those choices are invisible when the agent produces a correct answer and catastrophic when it produces a confident wrong one. When you run five agents in parallel with five genuinely different angles — one probing at the network layer, one at the service layer, one reading the registry on disk, one inspecting the process tree, one reading the source — you don't just get five data points. You get five chances for two agents to directly contradict each other, and that contradiction is more informative than any individual answer. A single agent probing a broken hostname-resolution layer will conclude a node is unreachable. Two agents — one using the hostname, one using a raw IP address — will contradict each other and point directly at the broken layer. The contradiction is the diagnosis.

The second mechanism is adversarial verification. Parallel agents can be tasked not to produce findings but to attack them. Before any irreversible action — publishing, deploying, sending, committing a claim to memory — one agent's job is to find reasons the main agent's output is wrong. Not to help, not to refine, but to refute. This is not the same as asking an agent to review its own work; it is structurally different. An adversarial sub-agent that finds nothing wrong is genuine evidence. The same agent that produced the claim searching its own work for flaws is not. The asymmetry matters because irreversible actions have asymmetric costs.

The third mechanism is idempotent declare-then-reconcile. A team of agents is not an ephemeral arrangement of running processes. It is a specification: a charter that says who is on the team, an engine map that says how to spawn each member, and a live-state file that says what is running right now. The first two files are durable and version-controlled. The third is disposable — it is the output of reconciling the declaration against the current machine state. Every operation that instantiates a team should be a re-runnable move toward the declared configuration. If a pane is missing, the operation creates it. If the declared isolation is not true, the operation does not patch around it — it destroys and recreates. A declaration is a request; reconciliation is the proof that it was honored.

These three mechanisms are independent of any particular tool. They do not require a specific agent runtime, a specific model vendor, a specific orchestration framework, or a specific host. They require only that you actually implement them, rather than assuming they emerge from running multiple agents at the same time.

Who This Book Is For

Engineers who are already running one agent and want to understand when and why to run several — not as a strategy for parallelism, but as a strategy for correctness. Practitioners who have found that agent outputs can look confident and be wrong, and want structural defenses against that, not just better prompts. Teams building orchestration layers who want durable patterns, not vendor-specific recipes.

How to Read It

The first four chapters establish the core mechanisms and the failure modes they address. Read them in order. Chapters five through eight build the operational infrastructure: how to specify teams as files, how to gather results, how to verify the right properties, and how to isolate parallel writers. The final four chapters cover the meta-patterns: discovery strategies, orchestration topologies, synthesis discipline, and when to stop abstracting. Each chapter stands on its own for reference, but the arguments build on each other.

1. Why Parallel Agents

A single agent cannot see its own blind spots. Parallel agents with different methods can.

A sequential agent makes choices. Every time it decides how to probe a system, which file to read first, which API endpoint to call, which metric to treat as ground truth — it is making a methodological choice. Those choices compound. By the end of a run, the agent has constructed a coherent picture of the world, but the coherence is partly a product of its own method. It found what its method was capable of finding.

This is not a bug in the agent. It is a structural property of any single observer.

The problem is that the agent has no way to know what it missed. It completed its checks. Nothing errored. It returns a result with confidence. From inside a single execution path, there is no signal that the method itself was the source of the answer.

Parallel agents using different methods break that feedback loop.

The Consistency Problem

Imagine asking one person to audit a distributed system. They pick a starting point, a toolset, a set of assumptions. They work through the problem and produce a report. That report will be internally consistent — which is exactly why you should not fully trust it.

Internal consistency is not the same as correctness. A consistent report means the observer's assumptions agreed with each other. It does not mean the assumptions agreed with reality.

Now give the same task to three people who have never compared notes, who use different tools, and who approach the problem from different angles. Their reports will probably disagree somewhere. That disagreement is not a failure — it is the most valuable output of the exercise. The disagreements mark the places where the answer depends on how you asked the question.

The same principle applies to software agents. A single sequential agent is coherent. A set of parallel agents with different methods is more likely to be correct, because their disagreements reveal where methodology was driving the result.

The Health Check That Wasn't

Here is a concrete example. A team of five agents was deployed to run an infrastructure health check against a cluster of services. Each agent was given the same list of hosts and the same pass/fail criteria. The agents differed in one key dimension: how they resolved and contacted each host. Some used hostnames. Some used raw IP addresses. One used a service mesh that handled resolution internally.

One host came back as DOWN from three agents and UP from two.

A single sequential agent would have picked one resolution method, gotten one answer — probably DOWN, since three out of five independent samples would have pointed that way — and reported the host as down. There would have been no reason to doubt the result. The agent completed its check. The check had a clear answer.

But because five probes ran simultaneously, the disagreement was visible immediately. The two UP results were not noise — they came from the agents using raw IP addresses, which bypassed the hostname-resolution layer entirely. The three DOWN results came from agents resolving through DNS, where a stale or misconfigured record was returning an address that was no longer valid.

The host was healthy. The DNS record was broken.

A single probe would have diagnosed a dead host. The actual problem was a dead DNS record. Those lead to completely different remediation paths: one has you restarting services, the other has you editing a zone file. Running a single sequential probe would not just have produced an incomplete answer — it would have produced a misleading answer that pointed engineering effort in the wrong direction.

The parallel run did not just find the bug faster. It found a different bug entirely.

What "Different Methods" Means in Practice

Methodological diversity does not require agents to be fundamentally different. Small variations are enough to expose inconsistencies.

In the health check example, the only variation was in how each agent resolved a hostname. That one degree of freedom was sufficient to reveal a DNS fault that would have been invisible to any single consistent probe.

In a code review context, the same mechanism applies. One agent reviews a patch against the stated requirements. Another reviews it against the existing test suite. A third looks for resource leaks and error handling. Each is a coherent review on its own. Together, they triangulate — and when they disagree on whether the change is safe, the disagreement tells you exactly where to look.

The point is not to run ten agents and hope one of them finds something. The point is to design the methodological variation deliberately, so that each agent's blind spots are covered by another agent's field of view.

Why Sequential Doesn't Catch This

The obvious objection: couldn't a single agent just run multiple methods sequentially?

In principle, yes. In practice, the agent that runs method A first will already have a working hypothesis when it runs method B. If method B produces a different result, the agent faces a choice between revising its hypothesis and rationalizing the discrepancy. Agents — like humans — have a strong pull toward the hypothesis they already hold. The first result anchors everything that follows.

Parallel execution does not have this problem. No agent has seen the others' results when it forms its own answer. The independence of the observations is structural, not a matter of discipline.

This is why parallelism is not just a performance optimization. It is an epistemic tool. Running agents in parallel is a way of getting independent observations on the same system — observations that can be compared, crossed, and used to identify where the answer is a function of the method rather than a function of reality.

The Shape of This Book

The rest of this book is about building systems that exploit this property deliberately: how to design agent teams with structural methodological diversity, how to aggregate their outputs without losing the signal in disagreements, how to handle failure modes that only appear in parallel execution, and how to reason about the reliability of a multi-agent result versus a single-agent result.

But all of it rests on this foundation: the value of parallel agents is not speed. Speed is a side effect. The value is that independent methods, run simultaneously, reveal what any single method structurally cannot see.

The rule: never trust a single agent's confident answer on any system where the method of asking could change the answer — run at least two independent probes with different approaches, and treat their disagreement as the most important part of the output.

2. Fan Out With Distinct Angles

Parallelism is only useful when each agent is asking a different question.

The naive version of multi-agent parallelism is five workers all running the same search. You get five answers. They agree or they disagree. Either way you have learned approximately nothing you could not have learned from one. The agents were parallel in execution; they were not parallel in coverage.

The useful version is N agents, each assigned a non-overlapping slice of the problem space. When they finish, the union of their outputs is larger than any single agent could have produced. That is the whole mechanism. Diversity of method is not a nice-to-have — it is the point.

What "Distinct Angles" Actually Means

An angle is a combination of: the data source you query, the tool you invoke, and the question you are trying to answer. Two agents have distinct angles when changing one of those three things would materially change the answer.

"Search documentation for X" and "search documentation for Y" are not distinct angles. They use the same source and the same tool; they just parameterize differently. A single agent could do both sequentially in seconds.

"Search documentation for X" and "read the live process table for X" are distinct angles. One looks at declared intent; the other looks at observed behavior. They can and do disagree, and that disagreement is information.

The practical test: if the two agents could swap instructions mid-run and produce equivalent output, they are not distinct.

Angle Taxonomies by Domain

Angles are domain-specific. Here are three common shapes.

Research. Partition by source type: primary sources (original papers, RFCs, specs), secondary analysis (synthesized commentary, blog posts), structured databases (version tables, vulnerability registries), and informal practitioner writing (forums, mailing lists, issue trackers). A research lead assigns one agent per tier. The outputs differ by epistemic status, not just content, which is why the compilation step is non-trivial.

Infrastructure diagnosis. Partition by layer: network reachability, service-level response (HTTP, RPC), configuration on disk (registry files, manifests), live process state (what is actually running), and source-of-truth reads (what the codebase says should be running). Each layer has a distinct failure mode. A network failure looks like silence from all other layers; a misconfigured registry looks like a healthy network but a broken service. You cannot infer layer N from layer N−1.

Build or refactor. Partition by component. One agent owns the persistence layer, one owns the API surface, one owns the front-end state model. They operate on different files, commit to different branches or worktrees, and block each other only at integration. The lead's job is sequencing the merge, not supervising the work.

Proof: A Five-Layer Fleet Map

A five-agent health probe ran against a fleet of machines connected by a mesh VPN, each running a collection of services registered in a shared registry. The question was: what is actually running, and does it match what should be running?

One agent pinged the network layer — pure reachability, no application protocol. It returned which hosts were visible at the transport level.

A second agent probed the service endpoints on each visible host via HTTP — health checks, version strings, response latency. It returned which declared services were responding.

A third agent read the registry on disk — the configuration files that described what each host was supposed to run. It returned the declared state.

A fourth agent queried the live process roster on each host via shell. It returned what was actually executing, including processes that had no registry entry.

A fifth agent read the relevant section of the transport source — the part of the codebase that defines how services register and deregister themselves. It returned the authoritative rules for what a valid registration looks like.

The lead compiled the five outputs into a single table: expected vs. declared vs. live. The result was a complete fleet map that no single layer could have produced:

Two hosts were network-reachable but had no registry entries (discovered by comparing agent one and agent three).
One registered service was returning HTTP 200 but the process roster showed the wrong binary version (discovered by comparing agents two and four).
The source agent revealed that a registration field used by several hosts was deprecated and silently ignored by the runtime — meaning the registry entries were valid-looking but semantically dead.

None of these findings were reachable by repeating any single layer's query. The value came entirely from the joins across distinct outputs.

The Failure Mode: Angle Collapse

Angle collapse happens when instructions drift toward similarity at authoring time. A lead writes five tasks, each phrased slightly differently, but each bottoming out in the same operation: semantic search against the same index. The agents fan out, run in parallel, and return five variants of the same answer. The lead spends time synthesizing output that contains no new information.

The fix is to write the task list as an explicit inventory of sources and tools, not an inventory of questions. Before spawning, ask: which data source does each agent own? If two agents share a data source and a tool, merge them into one.

A second failure mode is premature sharing. Agents that can observe each other's partial results mid-run will converge. The lead sees a tidy consensus that hides the variance that would have revealed the real finding. Keep agents blind to each other until the compilation step.

The Compiler Role

The lead agent is not a voter. It is not averaging the outputs or picking the plurality answer. It is a compiler: it takes outputs with known provenance (this came from the network layer; this came from the source) and constructs a representation that is only possible because the inputs were generated independently from different angles.

Compilation requires knowing what each agent was looking at. The task assignment must carry that provenance, not just the result. An agent that returns "service X is healthy" is less useful than one that returns "service X returned HTTP 200 from the health endpoint at layer 2; process roster was not checked." The qualifier tells the compiler where the finding fits in the join.

The rule: assign agents by data source and tool, not by topic — if two agents could swap tasks and return equivalent output, they are not distinct enough to parallelize.

3. Contradiction Is the Signal

When two agents disagree, the disagreement is the answer.

Parallel agents are often valued for their speed — fan out, gather results, move on. But speed is the secondary benefit. The primary benefit is that independent probes of the same thing, using different methods, will occasionally produce contradictory results. That contradiction is not noise to suppress. It is a diagnostic signal that localizes a fault with more precision than any single probe could achieve.

This chapter is about reading that signal correctly.

The Shape of the Problem

Every system has layers. A high-level command sits above a protocol, which sits above a transport, which sits above hardware. Each layer adds abstraction and adds the possibility of drift — a moment where the abstraction's view of the world diverges from the reality beneath it.

A single probe takes one path through those layers. Whatever it reports, you accept. A pair of probes taking different paths will agree almost always — and when they disagree, they have triangulated the layer where the drift lives.

Three real cases from a single debugging run illustrate this precisely.

Case 1: The Stale-Cache Ghost

A five-agent health probe was dispatched to assess the state of a service mesh. One agent queried a high-level status command — the orchestration tool's native "is this node up?" interface. It returned: offline. A second agent opened a raw TCP connection to the same host's HTTP port and sent a minimal request. It received a 200 response in 47 milliseconds.

The service was not offline. The status command was reading a cached state record that had not been refreshed since a restart cycle completed. The cache TTL had been set aggressively and the record had expired in the wrong direction — stale "offline" rather than stale "online" — because the restart had momentarily flipped the state and the next refresh never ran.

The contradiction localized the bug to one layer: the cache. Without the second probe, the orchestrator would have routed around the node, logged it as unavailable, and moved on. The ghost would have gone undetected.

Case 2: The Name That Pointed Nowhere

In the same run, two agents attempted to reach a different host. The first resolved the hostname through the normal name-resolution stack and attempted a connection. It failed: connection refused. The second agent skipped the resolver entirely, used the raw IP address that appeared in an inventory file, and connected successfully.

The host was alive. The name-resolution layer — a mesh VPN's internal DNS — had lost its record for that host. The hostname pointed nowhere, or pointed to an old address that no longer answered. The raw IP still worked because the underlying network had not changed; only the naming layer had drifted.

This is a common failure mode in dynamic infrastructure. Hosts are renamed, moved, or re-registered. The naming service is supposed to track those changes. Sometimes it doesn't. A probe that trusts the name will fail; a probe that bypasses it will succeed. The disagreement tells you exactly where to look.

Case 3: The Registry That Had Stopped Watching

A registry file on disk listed a set of peer agents and their last-seen timestamps. Several entries were marked stale — the timestamps were days old. An agent reading the registry concluded that most of the fleet was down. A separate agent sent live HTTP pings directly to each listed peer using the addresses in the registry. Every one responded.

The registry's refresh cycle had stalled. The process responsible for updating timestamps had stopped writing without raising an error. The registry's content was structurally valid, internally consistent, and entirely wrong about the current state of the fleet.

Again, one method read an abstraction — the registry — and got a false picture. Another method bypassed it and touched reality directly. The contradiction said: the registry is the problem.

How to Read the Contradiction

All three cases share a structure. One probe trusts an abstraction layer — a cache, a name resolver, a registry. Another probe works below or around that layer. When they disagree, the abstraction has drifted from the underlying reality, and the probe that bypassed the abstraction is reporting ground truth.

The resolution rule follows directly: trace the contradiction to its source. Do not average the results. Do not majority-vote across five agents where three of them all happened to use the same broken layer. Do not pick the answer that matches your prior expectation. Pick the answer that came from the more direct method, investigate why the indirect method disagrees, and fix the gap.

This requires discipline. The natural instinct is to trust the high-level tool — it is the authoritative interface, it was designed for this purpose, it is what experienced operators use. But "authoritative" means nothing when the authoritative source is stale. The inconvenient raw-socket result that contradicts it is not wrong; it is more right, because it is closer to the physical fact.

Designing for Contradiction

This is not an accident of the cases above. It is a design principle. When you orchestrate parallel agents, give them different methods on purpose. Do not send five agents that all query the same status API — you will get five copies of the same cached answer. Send one to query the status API, one to probe the port directly, one to tail the log, one to read the process table, one to ping from outside the network. If they all agree, you have high confidence. If any of them disagree, you have a localized lead.

The disagreement is the value. A single probe can fail silently — it takes one path, reports one answer, and you never know whether that path was reliable. Multiple probes with different methods create the conditions for contradiction to surface, and contradiction carries information that agreement cannot.

A system that never produces contradictions has not proven it is consistent. It has only proven that all its probes share the same blind spot.

The rule: when two independent probes return opposite answers, do not reconcile — investigate; the contradiction has already told you which layer to look at.

4. Adversarial Verification

Don't trust a finding because it is plausible; spend agents trying to kill it.

Plausibility is not evidence. An agent that returns a confident diagnosis has done one thing: it has found an explanation consistent with the observable facts. That is not the same as finding the correct explanation. The gap between those two is where production incidents live.

Adversarial verification is the practice of allocating agents not to confirm a finding but to refute it. The structure is deliberate: the default verdict is "refuted," and the burden of proof falls on the original claim. If a majority of the verification agents cannot be convinced, the finding dies. This feels wasteful until you watch it catch the third error the authoring agents missed.

Refute-by-Default Skeptics

The most common mistake in multi-agent verification is running N identical reviewers and averaging their confidence scores. You get correlated noise. If the first agent made a category error, the next four will make the same one because they are reading the same evidence with the same priors.

Invert the stance. Each skeptic's job is not to evaluate whether the claim is true; it is to find one specific reason the claim is false. Assign the verdict to "refuted" at initialization. Flip it to "confirmed" only if the skeptic exhausts its lens without finding a refutation. Aggregate across skeptics by majority: if more than half cannot refute it, the claim survives.

The consequence of this framing is that a single strong refutation from one narrow lens can kill a finding that nine other agents rated as probable. That asymmetry is intentional. Irreversible actions — publishing, deploying, sending — should require a high bar. Reversible actions can tolerate a lower one.

Perspective-Diverse Verifiers

Diversity of stance must be structural, not requested. Asking agents to "consider multiple perspectives" in the system prompt does not produce diversity; it produces a single agent hedging. Diversity requires that each verifier receive a distinct failure-lens as its sole mandate.

Useful lenses for a software context:

Correctness: Does the output match the specification under the inputs given?
Security: Does this change introduce an injection surface, a privilege escalation path, or an information leak?
Reproducibility: Can I follow these steps on a clean environment and arrive at the same result?
Regression: Does this break anything that was previously true?
Boundary conditions: What happens at the edges — empty input, maximum cardinality, clock rollover?

Five agents, five lenses. Each one is a specialist with a short brief and a binary output. The orchestrator collects verdicts and applies the majority rule. An agent that returns "cannot refute" after genuinely trying is contributing real information. An agent that returns "refuted" with a specific mechanism is contributing more.

The Single-Mandate Review Gate

Before any irreversible action, insert a gate: one agent whose only job is to scan for a single, well-defined failure class. Not a general review. Not a quality pass. A narrow, specialized search.

The proof is concrete. A team of writing agents produced a document intended for public release. The authoring agents, the structural reviewers, and the style editors all passed it. Before publication, a single gate agent received one instruction: find any string in this document that looks like a private identifier — an internal username, a server hostname, an API key fragment, a path beginning with a company-internal prefix. It found three. The authoring agents had paraphrased around them, reducing their visibility, but had not eliminated them. The gate agent, running nothing but a pattern-and-heuristic scan with that single mandate, caught what the broader review missed.

The lesson is not that authoring agents are careless. The lesson is that agents optimizing for quality of prose are not simultaneously optimizing for absence of leaks. Those are different objective functions. Give each objective to a different agent.

An Agent That Killed Its Own Hypothesis

The more instructive proof is when the verifier turns its skepticism on itself.

A five-agent health probe was investigating a silent failure in a distributed service mesh. One agent formed an early hypothesis: the failure was a name-resolution problem. The service could not find its upstream. The agent built a case — logs showed connection timeouts, which are consistent with DNS failures; the deployment had changed network topology recently; other failures in the fleet had been caused by resolution misconfiguration.

A well-incentivized agent treats its own hypothesis as a claim subject to the same adversarial treatment as any other. This one did. It ran resolution checks directly. Resolution succeeded. Every query returned the correct address with normal latency. The original hypothesis was not just unproven — it was actively falsified by the agent's own test.

Rather than preserving the explanation, the agent marked the hypothesis refuted and re-scoped. It looked for what else could produce connection timeouts when addressing was correct. It found a probe with no retry budget. The upstream was intermittently slow during cold starts; the probe hit the slow window, got a timeout, and reported failure. One-line fix.

If the agent had been built to defend its initial claim — if its reward signal came from having a hypothesis confirmed rather than from finding the true cause — it would have spent cycles explaining away the successful resolution test instead of discarding the hypothesis. The incentive structure determines whether an agent can discard its own work. Build it so that the honest answer, even when it is "I was wrong," is the rewarded answer.

Building the Incentive Right

Adversarial verification is not a prompting trick. It is an architectural choice. You need agents whose output is a binary verdict with a mechanism, not a confidence score with hedges. You need a majority rule that gives a single strong refutation the power to kill a finding. You need gate agents so narrow in mandate that they cannot rationalize away the one thing they are looking for. And you need authoring agents that are not the same agents doing the verification.

The agent that writes the code should not be the agent that reviews it for security. The agent that formed the hypothesis should not be the one that tests it — or if it must be, the test must be designed to falsify, not to confirm.

The rule: Default the verdict to refuted, diversify the lenses, and build every agent so that "I was wrong" is as valid an output as "I was right."

5. The Team Is a File

A running team of agents looks like a process; treat it like a document and you get reproducibility for free.

Every team of parallel agents encodes three categories of fact. The mistake most builders make is treating all three facts as one thing — dumping them into a single config blob and calling it done. The categories change at different speeds, are owned by different parties, and have completely different durability requirements. Conflating them is why teams are hard to reproduce and easy to lose.

Name the three categories and the architecture becomes obvious.

WHO and WHAT: The Charter

The charter is a human-authored recipe. It names the roles on the team, describes the intent of each role, and sets the boundaries of what the team is for. A coding-review team might have a proposer, a critic, and a summariser. A five-agent health probe might have one scout per service endpoint and an aggregator that waits for quorum. The roles are stable. They change when the human decides the team's job has changed — not when a process crashes, not when you switch machines, not when you upgrade a runtime.

Because the charter encodes intent, it is irreplaceable. You cannot reconstruct it from a snapshot of running processes. If you lose the charter, you lose the knowledge of why the team exists and what invariants it is supposed to uphold. This is the fact that must live in version control.

The charter is also the only file a human should need to read to understand what a team does. Keep it short. One role per stanza, one line of intent per role, explicit preconditions if any. It is a recipe, not a manual.

HOW: The Engine Map

The engine map is a dictionary. Each entry maps a friendly runtime name — a short token you use throughout the charter and the orchestration tool — to a literal launch command: the shell invocation, the flags, the environment variables that boot exactly one agent in exactly one runtime.

The engine map is the only cross-runtime file in the system. One entry might boot a frontier model from vendor A; the next might boot a different vendor's runtime entirely. The map is what lets you write a charter that says role: critic, engine: fast-reviewer without encoding a vendor decision into the charter itself. You swap the engine map entry and the charter stays clean.

Like the charter, the engine map belongs in version control. It changes when you decide to switch runtimes or tune launch parameters — a deliberate human decision, not a runtime event.

NOW: The Live State

The live state is written by the system, not by a human. When the orchestration tool boots the team, it records the pane identifiers assigned by the terminal multiplexer, the process IDs, which members are currently alive, their working directories, any inter-agent message-bus addresses. This is a snapshot of this instantiation of the team on this machine at this moment.

The live state is meaningless anywhere else. The pane identifiers are local to one multiplexer session. The process IDs are local to one OS. The working directories are absolute paths on one filesystem. Committing the live state to version control is noise at best and misleading at worst — a teammate on a different machine would get a file that looks authoritative but describes a world that no longer exists.

The live state is also fully derivable. Given a charter and an engine map, the orchestration tool can re-boot the team from scratch and write a fresh live state. The derivability is the entire point: it means you can treat the live state as a disposable cache, not a source of truth.

The Proof: A Dead Snapshot

A team of five coding agents was running a long refactor across parallel worktrees. The terminal multiplexer session died — power loss, not a graceful shutdown. The live-state file survived on disk, listing pane IDs, process IDs, agent statuses.

Re-attaching to those pane IDs failed silently: the multiplexer session was gone, so the IDs referred to nothing. Attempting to send a message to the recorded process IDs produced no output and no error. The live-state file looked like a map, but every address on it was a dead letter.

Recovery took thirty seconds. The orchestration tool read the charter and the engine map — both in version control, untouched — and re-booted the team into a fresh multiplexer session. The new live-state file was written, the agents picked up their worktrees, and the refactor resumed. Nothing was lost except the in-flight work that had not been committed, which is a git problem with a git solution.

If the charter had not been in version control — if it had been stored only in the live-state file, or in the session state of the multiplexer — recovery would have required reconstructing intent from running processes that no longer ran. That is not a recoverable situation.

The Asymmetry Is the Architecture

The three files have different volatility signatures. Charter: low volatility, human-authored, git-committed. Engine map: medium volatility, human-decided, git-committed. Live state: high volatility, machine-written, git-ignored.

The asymmetry is not an accident of implementation. It is the load-bearing property of the design. You can always regrow the live state from charter plus engine map. You can never reconstruct the charter from a dead snapshot. That single directional dependency — live state derives from the other two, not the other way around — determines what to commit and what to discard.

When you add a new role to the team, edit the charter. When you swap runtimes, edit the engine map. When the team crashes and you need to restart, delete the live-state file and let the orchestration tool regenerate it. Three files, three ownership models, three commit policies.

The rule: commit what humans author and machines cannot reconstruct; discard what machines write and can always regenerate.

6. Ensure and Gather

Two idempotent verbs are all you need to keep a local agent team honest.

The daily friction of running a local agent team is not configuration, not model selection, not prompt engineering. It is drift: the gap between the team you declared in a file and the processes that are actually running on your machine. Agents crash. You close a terminal. A relaunch lands in the wrong window. Within an hour of a fresh spawn, the declared team and the live team have diverged, and you are flying blind.

The fix is two orthogonal, idempotent verbs: ensure and gather.

Ensure: Close the Lifecycle Gap

Ensure answers one question: for every member declared in the live-state file, is a live session running?

The algorithm is a three-way classify-and-act loop. For each declared member, probe the agent runtime for a session with that member's identifier. The result is one of three states:

Live — session exists and is responsive. Skip. Do nothing.
Missing — no session exists at all. Spawn a new isolated session with that member's config.
Dead — a session record exists but the process has exited. Relaunch in place, reusing the existing session slot so the identifier stays stable.

That three-way branch is the entire ensure operation. Notice what it does not do: it never touches a live session. If you run ensure on a healthy, fully-staffed team, zero processes are disturbed. That is the idempotency guarantee. You can run ensure from a cron job, from a pre-task hook, from the top of every orchestration script — the cost on a healthy team is a few milliseconds of probe round-trips.

The missing-versus-dead distinction matters. Missing means spawn fresh: a new session, new environment, new working directory per the charter. Dead means the slot existed — the orchestration tool may still hold metadata, scroll buffer, window position — so you relaunch inside it rather than creating a duplicate. Skipping that distinction causes duplicate entries in the registry, which then require manual cleanup.

Spawning must be isolated. Each member gets its own session, not a pane in a shared window. The reason is robustness: a shared window means a single terminal multiplexer crash takes out multiple agents. Isolated sessions fail independently.

Gather: Compose Without Restarting

Ensure gives you a correctly staffed team. Gather gives you a view of it.

The default state after ensure is a set of isolated sessions scattered across detached terminal multiplexer windows — invisible, uncoordinated to look at. Gather pulls them into your current window as a tiled layout of panes. Scatter is the inverse: it breaks the panes back out into isolated windows, returning the team to the detached state you started from.

The key mechanism: relocating a running pane is a move, not a restart. The process keeps executing through the relocation. A coding agent mid-way through a file edit does not notice that its pane was joined to a layout. This makes placement operations essentially free, and it makes gather re-runnable at any time — you can gather, inspect, scatter, and gather again without disrupting any agent's work.

Two failure modes bite you if you implement this naively.

Address invalidation. When a pane moves from its original window into your gather window, the old address — the window-and-pane index the orchestration tool used to target that member — is no longer valid. Gather must re-resolve the address of every relocated pane and update the registry entries accordingly. At the end of a gather operation, print the current send-command for each member so downstream scripts do not cache stale addresses. Scatter has the same obligation in reverse.

Window name loss. When you break a pane out of a window into a new standalone window, the terminal multiplexer assigns it a default name, usually the current process name or a generic placeholder. The original window name, which encodes the member identifier, is gone. The fix is to capture the window name before any break-out operation and re-apply it immediately after. Two lines of code, easy to miss, painful to debug — you end up with five windows all named "zsh" and no way to know which agent is which without reading process output.

Composing the Verbs

Ensure and gather are orthogonal by design. Ensure is pure lifecycle; it knows nothing about pane layout. Gather is pure placement; it assumes all sessions already exist. That separation means each verb stays simple and each can be tested independently.

They compose cleanly: a flag on ensure — call it --gather — runs gather immediately after the lifecycle pass completes. This is the one-shot "make my team real and put it in front of me" command that you end up running at the start of every session.

A concrete example of this in practice: a five-agent health-probe team, each agent responsible for a different service cluster, runs autonomously during off-hours. In the morning, two agents have crashed on timeout errors, one is missing entirely because the session was never relaunched after a machine sleep, and two are live. Running ensure --gather spawns the missing agent, relaunches the two dead ones, leaves the two live agents untouched, then tiles all five into the current window. Total elapsed time: under four seconds. No agent that was mid-task was interrupted.

Keeping the Verbs Honest

Both ensure and gather must read from the live-state file, not from memory or cached state. The live-state file is the declared reality; everything else is derived. If you start caching member lists in your orchestration script, you will eventually run ensure against a stale list and miss a member that was added by another session.

Ensure should log its actions — spawned, relaunched, skipped — at a single line per member. Silence on a live member, a line on any gap. That log is the audit trail that tells you, the next morning, what the team looked like when it reconvened.

Gather should be fast enough to run reflexively. If it takes more than a second on a team of ten, the address-resolution step is probably doing unnecessary round-trips. Probe once, batch the moves, re-resolve once.

The rule: Separate lifecycle from placement — ensure closes the gap between the file and the running process, gather composes the running processes into a view, and neither verb ever restarts what does not need restarting.

7. Verify the Invariant, Not the Roster

The declaration tells you what you intended; the running process tells you what exists.

You've assembled the team. The registry shows five agents, each with a name, a role, and a status light that reads "active." The charter file lists their responsibilities. The engine map says which runtime each one runs. Everything looks correct. You are about to begin distributing work.

Stop. You have verified the roster. You have not verified the invariant.

The distinction matters more than it first appears, and the cost of ignoring it compounds with every agent you add.

Declarations Are Not Evidence

A roster is a record of intent. It describes what should exist: which agents were spawned, which directories they were assigned, which runtimes they were handed. A declaration can be perfectly accurate at the moment of creation and silently wrong ten seconds later. The agent crashes. The pane survives. The status field doesn't know.

The invariant is the property you actually need: each agent is running its designated runtime, in its designated directory, processing work. That property must be read from the running world, not inferred from the config that described the world before it started.

This isn't a paranoid edge case. It's the normal failure mode of any system that hands you a view derived from events rather than from continuous inspection.

The Liveness Trap

Every terminal multiplexer ships an activity field. It is typically computed from recency: if the pane produced output within some window, it's "active"; beyond the threshold, it degrades to "idle" or "stale." This field is the obvious proxy for liveness and it is wrong.

Consider a coding agent deep in a long compilation or waiting on a slow API call. No output. Six minutes pass. The multiplexer marks the pane stale. That agent is alive, working, blocked on I/O — and indistinguishable, by activity timestamp alone, from a pane where the runtime exited and dropped to a bare shell. The activity field conflates "not printing" with "not running." Those are different facts.

The reliable signal is the pane's current foreground process. Query it directly. If the foreground process is the agent runtime, the agent is alive. If the foreground process is a bare shell, the runtime exited. That distinction does not fade with time. It does not lie about agents that are thinking quietly. It is a present-tense fact about the operating system's process table, not a timestamp derived from past behavior.

The implementation is straightforward: for each pane, resolve its foreground process group, walk the process tree, and check the executable name against the expected runtime. Five lines of shell. If it matches, the agent is live. If it doesn't, the agent is gone regardless of what the status field says.

The Silent Clobber

Activity status is the obvious failure. Working directory is the subtle one, and it can destroy correctness while the roster looks perfectly healthy.

Here is the proof as it happened. A team of three coding agents was assigned to work in parallel, each in a separate git worktree. The worktree pattern is the correct approach: isolated checkouts, no shared index, no accidental cross-contamination. The orchestration tool reported all three agents active. The layout looked tidy. Status: healthy.

A five-point health probe queried each agent's actual working directory by reading the process's current directory from the operating system — not from the config, not from the spawn command, from the live process. All three agents reported the same path. They were all working in the same worktree.

The spawn logic had a bug: the directory argument was being constructed before the worktrees were fully initialized, and all three resolved to the same default. The orchestration tool didn't notice because it never checked. The roster showed three distinct entries. The world contained three agents clobbering each other's work in one directory, racing on every write.

The roster lied. The working directory couldn't.

What to Inspect

Given that declarations are not evidence, what constitutes a real health check before you begin distributing work?

Process identity. Is the expected runtime the foreground process? Not "was it launched," not "has it printed recently" — is it running now?

Working directory. Is each agent in the directory it was assigned? This check takes one syscall per process. It catches the silent clobber. It costs nothing relative to the work you're about to hand out.

Uniqueness. If each agent is supposed to be isolated — in its own worktree, its own port, its own namespace — verify that the values are distinct across the team. A set of five working directories that collapses to three unique paths means two pairs of agents are sharing space they shouldn't.

Reachability. If agents expose an interface — a local HTTP endpoint, a socket, a named pipe — verify you can complete a round trip. A process can exist and be unreachable. A reachable endpoint proves more than a running process.

These are invariants. They describe properties the system must have for the work to be correct. Verifying them is not overhead; it is the proof that the system is in the state the roster claims it's in.

Inline, Not Postmortem

The natural instinct is to run these checks after something goes wrong. That's postmortem debugging, and by then you've distributed work into a broken environment, paid the cost of the failure, and spent time diagnosing something that was verifiable before you started.

Structural verification belongs in the startup sequence, before the first task is dispatched. It takes seconds. It converts a class of silent failures — wrong directory, dead runtime, duplicate assignment — into loud, early, actionable errors. The orchestration loop doesn't need to be clever. It needs to ask the operating system what's actually true, compare that to what should be true, and refuse to proceed if they don't match.

Announcements of readiness are cheap. Proof of readiness is a process query.

The rule: before distributing work to any agent team, verify each invariant — live process, correct directory, unique assignment, reachable interface — by inspecting the running system directly; a clean roster is necessary but not sufficient, and the gap between them is where silent failures live.

8. Isolation for Parallel Writers

Two agents, one directory, zero conflict markers — and half your work gone.

Parallel agents that only read can share whatever they like. Point three search agents at the same repository, the same log directory, the same knowledge base — they will never interfere. Read operations commute. Write operations do not.

The moment two agents write to the same working directory on the same branch, you have a clobber trap. Not a merge conflict. Not a clear error. A clobber trap is silent: the second writer's flush overwrites the first writer's, the file system records one version, and nobody complains. The only evidence is the missing work.

The Mechanism

A version-control working directory is a mutable snapshot. It holds one checked-out state at a time. When an agent edits a file, it modifies that snapshot in place. When a second agent edits the same file in the same directory, it modifies the same snapshot. Whichever process writes last wins. There are no conflict markers because there was no merge — just two sequential writes to the same inode.

The fix is structural, not procedural. You cannot solve this with careful scheduling or polite conventions. You solve it by giving each writing agent its own isolated working copy, on its own branch. Git worktrees are the clean primitive for this: a single repository can have multiple checked-out working trees simultaneously, each on a distinct branch, each in a distinct directory on disk. Writers cannot collide across worktree boundaries. The isolation is enforced by the file system, not by agent discipline.

Proof: The Shared-Directory Trap

A convenience spawn brought up a coordination agent plus two coding agents. The spawn was fast — all three appeared in a single command, roster printed cleanly, team announced ready. Coordination in one pane, Coder A in another, Coder B in a third.

All three landed in the same working directory, on the same branch.

The coordination agent was fine. It read tickets, wrote summaries to a scratch buffer, never touched source files. Read-only agents can share anything. The problem was Coder A and Coder B. Each had been handed a task that required editing source files. The moment both agents started writing — not if, when — one would overwrite the other. The file system would pick a winner based on flush timing, and the loser's work would disappear without a trace.

The team looked correct from the outside. Three members, three roles, a printed roster saying each member was ready and assigned. The declaration of readiness was accurate for the coordination agent. It was a fiction for the coders.

Why You Cannot Patch a Live Process's Birthplace

The obvious repair is to relocate the offending agents into their own worktrees. The problem: you cannot move a running process's working directory after the fact. A shell process inherits its working directory at spawn time. Its current directory is a kernel attribute of that process, and you cannot retroactively change it from outside the process — not cleanly, not portably, not safely for a live agent that may be mid-operation.

The repair for the shared-directory trap is therefore destructive: you stop the incorrectly spawned agents, provision separate worktrees, and respawn each writer into its own isolated ground. There is no patch-in-place option. This is not a flaw in the tooling; it is a consequence of how operating systems model process working directories.

This means isolation must be declared before spawn, not fixed after. The worktree must exist, the branch must be created, and the spawn command must target the correct directory — in that order, before the agent process starts.

The Proof Requires Observation, Not Trust

Declaring isolation in a config file or a spawn script is a request. It is not a guarantee. The only guarantee comes from observing the running system.

After a team of writers comes up, verify two properties before declaring the team ready:

One: distinct working directories. Enumerate each agent's working directory — query the process, the orchestration tool's agent registry, or whatever source of truth your runtime exposes. Assert that no two writing agents share a path. If two paths are identical, the team is not safe, regardless of what the config says.

Two: distinct branches. List the branch each worktree is checked out on. Assert uniqueness among writers. Two agents in separate directories but on the same branch is still dangerous if you later run a merge step that assumes branch-per-agent isolation.

A three-line verification that checks these two properties is worth more than any amount of spawn-time convention. Run it after every team bring-up. If the assertion fails, stop the team before any writes happen. Clobber damage is hard to detect and harder to recover; preventing it costs almost nothing.

Shared Directories Are Fine — for Readers

This is not a rule that all agents must be isolated from each other. Read-only agents — reviewers, searchers, report generators, anything that never modifies a file — can share a working directory freely. There is no cost to giving them a common checkout. The isolation requirement is scoped precisely: one worktree per writing agent, no exceptions.

A team of five where two write and three read needs two isolated worktrees (one per writer) and can share or not share for the readers. The constraint is proportional to the hazard.

Structure of a Safe Writer Spawn

The sequence that avoids the trap:

Create the branch for this agent's work.
Add a worktree for that branch in a dedicated directory.
Spawn the agent with its working directory set to that worktree path.
After spawn, verify: working directory matches the expected path, branch matches the expected name.
Record both in the team's live-state file so later stages can locate each agent's output.

Steps 1–3 are the setup. Steps 4–5 are the proof. Skip the proof and you are trusting declarations, which is how the shared-directory trap happens in the first place.

Summary

Shared state is safe for readers and fatal for writers. The structural fix is git worktrees — one per writing agent, provisioned before spawn. The process-working-directory constraint means you cannot repair this after the fact; you destroy and recreate. And a declaration of isolation in config is only a request: close the loop by observing the actual running directories and asserting uniqueness.

The rule: every agent that writes gets its own worktree and its own branch, provisioned before spawn and verified after — declaration is not proof, observation is.

When you don't know how big the answer is, count down to silence — not up to a quota.

The most dangerous assumption in agent orchestration is that you know the size of what you're searching for. Count the bugs in a codebase. Find every stale reference in a distributed config. Locate all the unhealthy nodes in a service mesh. In each case, you are looking for something whose population is unknown, and an arbitrary stopping criterion — "run five finders and collect results" — will miss the tail systematically.

Two patterns address this. Loop-Until-Dry handles depth: keep searching until the search demonstrably stops yielding. Multi-Modal Sweep handles coverage: search the same space several ways simultaneously, because no single approach finds everything.

Loop-Until-Dry

The structure is simple. Spawn a round of finder agents. Dedup their output against a global SEEN set. If the new-findings count is above zero, start another round. If K consecutive rounds surface nothing new, stop.

The dedup rule is load-bearing. You must dedup against everything ever surfaced, not just everything that survived downstream review.

Here is why that matters. Suppose a finder surfaces a candidate. A judge agent reviews it and rejects it — not a real bug, false positive. On the next round, another finder surfaces the same candidate. If SEEN only tracks accepted findings, this candidate is new to SEEN and gets routed to the judge again. The judge rejects it again. Round after round the same false positives circulate, the new-findings count never reaches zero, and the loop never converges.

The fix: add to SEEN at intake, before review. The judge still operates on the full set, but the loop's convergence signal is deduplication-before-review. Rejected items stay in SEEN with a rejected tag. They never appear in output. They do appear in the convergence check.

seen = {}           # keyed by canonical fingerprint
round_new = []

for each round:
    candidates = run_finder_wave()
    for c in candidates:
        key = fingerprint(c)
        if key not in seen:
            seen[key] = c
            round_new.append(c)

    if len(round_new) == 0:
        consecutive_empty += 1
    else:
        consecutive_empty = 0

    if consecutive_empty >= K:
        break

The fingerprint function matters too. For a bug report it might be (file, line_range, error_class). For a service-mesh node it might be (node_id, failure_mode). Sloppy fingerprinting — just the node ID, without the failure mode — merges distinct findings and makes the loop terminate too early.

Budget Guard

An open-ended loop needs a hard ceiling. The convergence signal is the primary brake; the budget guard is the emergency brake.

Track two numbers: rounds_elapsed and total_agent_calls. Set ceilings on both before the loop starts. When either ceiling is hit, the loop stops and logs a budget_exceeded event — so you know the search was truncated, not complete. Silence-after-K-rounds means done. Budget hit means unknown.

This distinction matters for callers. A downstream system that consumes "all live unhealthy nodes" should treat a budget-hit result differently from a clean-convergence result — perhaps escalate, perhaps widen the budget, perhaps flag the report as partial.

Loop-Until-Dry handles depth. Multi-Modal Sweep handles breadth. The premise: any single search strategy has blind spots, and the blind spots are often invisible from inside that strategy.

Consider finding every reference to a deprecated API in a large repository. Strategy A searches by symbol name. Strategy B searches by file-level import patterns. Strategy C searches by commit message for the migration flag. Strategy D searches by the test suite for test names that encode the old API name.

Run all four in parallel, with no inter-agent communication during the sweep. Each agent is deliberately blind to what the others find. They don't coordinate; they compete to find things the others miss. You collect results afterward and dedup into a merged set.

Why keep them blind during the sweep? Shared state during search creates anchoring bias. If finder B sees that finder A already found src/auth/token.go, B will unconsciously (or explicitly, if prompted carelessly) deprioritize similar files. The whole value of multi-modal sweep is that each strategy covers its own blind spots. Cross-pollinating during the sweep defeats this.

A Concrete Case

A team ran a sweep to find every misconfigured endpoint in a service registry — roughly four hundred services, each with several config fields. They dispatched five parallel finder agents: one searched by container label, one by config file pattern, one by named entity (service name prefix), one by last-modified timestamp window, and one by cross-referencing the live-state file against the charter file.

Round one surfaced sixty-one candidates. Round two, after dedup against the full round-one set, surfaced nineteen new ones. Round three: four. Round four: zero. Round five: zero. Two consecutive empty rounds was the configured K. Loop stopped.

Without multi-modal sweep, the label-only search would have found forty-three of those eighty-four. The timestamp-window search found eleven that no other strategy touched — services that had been renamed, breaking the label match, but the rename timestamp sat in the known migration window.

Without the dedup-before-review rule, the loop would have churned on eight false positives that the judge kept rejecting. Observed behavior in an early version: still running at round twenty-two.

Composition

The two patterns compose naturally. Run a multi-modal sweep as each round of a Loop-Until-Dry. Each wave dispatches N strategies in parallel; their merged output is dedupped into SEEN; the convergence check fires on the merged-and-dedupped delta, not per-strategy. Budget guards apply to the outer loop, not to individual strategies.

The structure looks like concentric loops: outer loop checks convergence, inner fan-out handles strategy diversity. Each layer has a single responsibility.

The rule: when the answer set has unknown size, measure by convergence — count down to silence across K empty rounds, dedup at intake (not after review), cover the search space from multiple independent angles, and let a hard budget guard distinguish "done" from "stopped early."

10. Orchestration Shapes: Pipeline vs Barrier

Two patterns govern every multi-stage parallel workflow. Choosing the wrong one doubles your wall-clock time for free.

Most orchestration mistakes are not logic errors. The code is correct, the agents do the right work, the outputs compose cleanly — and the job still takes three times as long as it should. The culprit is almost always an unnecessary barrier.

The Default: Pipeline

In a pipeline, each item moves through stages independently. Stage boundaries are not synchronization points; they are hand-off queues. Item A enters stage 2 the moment stage 1 finishes for item A, regardless of where items B, C, and D are.

The consequence for wall-clock time is significant. Suppose you have five items and three stages. Each stage costs roughly one second per item. In a pipeline, the last item exits stage 3 at approximately five seconds — one second of startup skew plus the slowest item's full chain. The stages overlap in calendar time. In a barrier version of the same job, you wait for all five items to clear stage 1 before any item starts stage 2, and again at stage 2 before stage 3. Wall-clock approaches three times the pipeline figure.

The formula is worth internalizing: pipeline wall-clock equals the slowest single item's full chain, not the sum of the slowest-per-stage. A barrier turns that sum back on.

This is why pipeline is the default. If you have no strong reason to synchronize, do not synchronize.

When a Barrier Is Justified

A barrier earns its latency cost in exactly three situations.

Cross-item deduplication before expensive downstream work. If stage 2 will invoke a costly operation — a long model call, a database write, a network fetch — and stage 1 can produce duplicates, collecting all of stage 1's output first and deduplicating the set before launching stage 2 is correct. You are paying barrier latency once to avoid paying downstream cost N times. The math only works if the downstream cost per duplicate exceeds the barrier latency; confirm that before adding the synchronization.

Early exit on empty set. If the count of stage 1 results is zero, there is no point starting stage 2 at all. You cannot know the count is zero until all of stage 1 has reported. A barrier here is not overhead — it is the gate condition.

Stage N must compare each item against the others. Some operations are inherently cross-item: ranking, clustering, consensus voting across agent outputs, finding the global maximum. You cannot rank item A until you have seen item B through item N. If stage 2 is "select the three highest-confidence results from the full set," it requires a barrier. There is no pipeline-compatible version of that operation.

Everything else is not a barrier condition.

What Is Not a Barrier Condition

Two false justifications appear repeatedly.

The first is "I need to flatten or filter the output before the next stage." Flattening a list-of-lists or filtering on a predicate is a map operation. It belongs inside a pipeline stage, not at a synchronization point. Each item's stage-1 output can be flattened and filtered as it arrives, and the result forwarded immediately into stage 2. Adding a barrier here serializes the pipeline for no reason.

The second is "the stages feel conceptually separate." Conceptual separation is about code organization, not execution order. A pipeline stage is already a discrete unit of work. Wrapping it in a barrier because it "feels like a different phase" imposes latency without buying correctness.

The test is strict: does stage N require data from all other items in stage N−1? If yes, barrier. If no, pipeline.

The Latency Cost Is Real

Consider a team of coding agents doing a two-stage job: first, each agent probes a set of endpoints to collect health data; second, each agent synthesizes a report from the collected data. Suppose the five probes take 4 s, 4 s, 4 s, 4 s, and 12 s respectively. Stage 2 takes roughly 3 s per agent regardless.

In a pipeline, the four fast probes hand off at 4 s and their synthesis runs in parallel with the slow probe's remaining 8 s. Total wall-clock: 12 s (slow probe) + 3 s (its synthesis) = 15 s.

With a barrier between stage 1 and stage 2, all synthesis is blocked until the slow probe finishes at 12 s. Then all five syntheses run in parallel, finishing at 15 s. Same number — but only because synthesis is uniform. If synthesis time also varied, the fast agents would be idle waiting for the barrier, then idle again waiting for the slowest synthesizer. The losses compound.

The general principle: if the slowest stage-1 item takes k times the fastest, a barrier wastes (k−1)/k of the fast agents' available time. At k=3, you waste two-thirds. That is not a rounding error; it is the dominant cost.

Composition

Pipelines and barriers compose. A multi-stage workflow might be:

pipeline → barrier → pipeline → pipeline

The barrier in the middle is where deduplication or a count check occurs. Everything else flows continuously. The shape of the workflow should match the data dependencies, not the programmer's intuition about "phases."

When designing a new multi-stage job, start by sketching data dependencies between stages: does stage N+1 need one item's stage-N output, or all items' stage-N output? The answer directly dictates the shape. One item's output → pipeline. All items' output → barrier.

The rule: default to pipeline; add a barrier only when a stage genuinely requires cross-item context from the complete previous result set, and verify that the downstream savings exceed the synchronization tax before you pay it.

11. The Synthesis Step

Fan-out is easy. Synthesis is where you earn the result.

Spinning up N agents in parallel is a solved problem. Give each one a prompt, a tool set, and a scope; wait for the futures to resolve; collect the results. Any competent orchestration layer can do this. The part that fails — quietly, in ways that look like success — is what happens next.

Most teams treat synthesis as concatenation. They append the N responses into a single document, maybe sort by confidence score, hand it to the user. This is not synthesis. It is a pile wearing a trench coat.

Contradiction Is a Signal, Not a Noise Floor

When two agents return different answers to the same question, the naive responses are:

Average the results (nonsensical for non-numeric claims)
Take the majority vote (correct on statistics, wrong on diagnosis)
Pick the answer that matches your prior expectation (confirmation bias dressed as engineering)

All three discard the most valuable information the run produced: the fact that two agents diverged at all.

Consider a five-agent health probe. Four agents report a host reachable; one reports it down. Majority vote says the host is up. But the one dissenting agent used a fully-qualified domain name while the other four used a raw IP address. The contradiction is not noise — it is a directed pointer at a broken name-resolution layer. The synthesis step that votes it down as an outlier has just hidden an incident.

The right behavior: flag the divergence, inspect the mechanism each agent used, surface "these two results are not comparable because they exercised different paths." That is thinking. That is synthesis.

The Compile Step Must Think

Think of synthesis as a compiler pass, not a text merge. A compiler doesn't concatenate object files; it resolves symbols, detects conflicts, and refuses to proceed if the graph is incoherent.

A synthesis agent's job is:

Deduplication. Multiple agents will surface the same finding through different framings. "The API returns 503 under load" and "the endpoint becomes unreachable at high concurrency" are the same claim. Merge them into one finding with both sources noted. Concatenating both creates false weight — the reader infers two independent observations when there is one.

Conflict resolution with trace. When two claims genuinely contradict each other, the synthesis step must not choose silently. It must record: what each agent found, what method it used, what source it read. Often the conflict resolves mechanically once the methods are compared (as in the DNS case above). When it doesn't resolve, the output should say so explicitly — "two agents reached opposite conclusions, here is why each is credible, here is what a tiebreaker run would need to examine."

Hierarchy of evidence. Not all agent results carry equal weight. An agent that read the primary source outranks an agent that inferred from secondary signals. The synthesis step should encode this: primary beats inferred, observed beats assumed, recent beats stale. A flat concatenation treats everything as equally authoritative.

Single merged truth. The final output should read as if one careful analyst did all the work. Not "Agent 3 says X, Agent 5 says Y." The attribution lives in footnotes or a provenance block; the main body presents the merged conclusion. If you cannot write a coherent merged conclusion, that is itself a finding: the run was insufficiently convergent and needs another round.

The Completeness Critic

Even a well-executed synthesis can be complete within its own frame while missing the frame entirely.

Add one more agent after synthesis: a completeness critic. Its prompt is not "is this correct?" It asks:

What modality was never exercised? (No agent tried the out-of-band path, the fallback endpoint, the read replica.)
What claim was stated but never verified? (The synthesis says "this service is stateless" — did any agent actually check for session state?)
What source was never read? (The charter file was summarized by an agent that didn't open it.)
What adversarial case was never attempted? (Every probe was sent from the same network segment.)

The completeness critic does not produce a result. It produces a list of gaps. Each gap becomes a work item for the next fan-out round. The loop closes.

This is the step most systems omit, because it requires acknowledging that the run was incomplete — which feels like failure. It is the opposite. A run that knows its own boundaries is more valuable than a run that confidently covers three-quarters of the problem space while presenting itself as comprehensive.

A Concrete Shape

A reliable synthesis pattern for a coding agent team looks like this:

Fan-out: each agent works a scoped problem and writes a structured result — findings, confidence, method, gaps it noticed.
Collect: wait for all results; record which agents timed out or errored (their absence is data).
Synthesize: one agent (or a deterministic function for simple cases) deduplicates, flags conflicts, traces each conflict to mechanism, produces a merged findings document with a provenance block.
Critic pass: one agent reads the merged document and the original task statement, then lists what is missing — no access to the original agent outputs, only to the synthesis and the task.
Branch: if the critic finds critical gaps, open a second fan-out targeting only those gaps; otherwise deliver.

Steps 3 and 4 are cheap — one or two model calls. The cost of skipping them is a subtly wrong answer that looks authoritative.

Why This Is Hard to Build

The temptation is to compress synthesis into the prompt of the final summarizing call: "Here are N results, summarize them." This works until it doesn't — until a contradiction slips through as a consensus, until a gap in coverage is smoothed over by confident prose, until the reader acts on a finding that two agents would have resolved to the opposite conclusion if you had compared their methods.

Synthesis is a structured process, not a single LLM call. It has typed inputs (findings with method metadata), typed outputs (merged truth plus provenance), and a separate critic with a separate prompt that has no stake in validating the previous step's work.

Build it as a stage, not an afterthought.

The rule: treat every contradiction between agents as a pointer at a mechanism difference, force the synthesis step to think rather than concatenate, and always run a completeness critic whose only job is to find what the run didn't cover.

12. Prototype Before You Abstract

Write the throwaway script first. The shape of the real feature lives inside it.

Every orchestration system eventually needs a reconciliation verb — something that looks at the live state of a running agent team, compares it against the declared intent, and closes the gap. The temptation is to design this as a feature: sketch the API surface, debate the data model, open a ticket, wait for the right sprint. Resist it. Build a thin shell script over the primitives you already have. Run it against the real system. Watch it break.

The breaks are the design.

Why a Script, Not a Design Doc

A design doc answers the question you know how to ask. A running script answers the questions the system actually has. These are not the same set.

When you write a reconciliation loop as prose — "fetch current state, diff against declared state, issue corrective commands" — the logic looks complete because the words are complete. The moment you script it, you discover that "fetch current state" means parsing output from three separate tools in three different formats, that the diff step silently swallows missing keys, and that "issue corrective commands" has a sequencing constraint nobody mentioned because everybody assumed somebody else had thought about it.

None of this appears in a design doc because design docs are written by humans who already understand the system. The script is written by a human who has to make the system do a specific thing by Tuesday, and that pressure surfaces every assumption.

The Five-Step Order

There is a sequence that works. Do not reorder it.

Declare the team. Before any script runs, the team must exist as committed text: a charter file that names the roles, an engine map that assigns each role to a model tier, a live-state file that records what is actually running right now. Three files, all readable, none implicit. This is the ground truth the reconciliation loop will read from and write to.

Script the reconciliation. Write a shell script — fifty to two hundred lines, no more — that reads those three files, calls the orchestration tool's existing primitives (spawn, stop, status), and prints what it did. No new abstractions. No helper libraries. Every operation visible on one screen.

Run it on the real thing. Not a staging environment, not a sandbox with mocked tool calls. The actual machines, the actual agent runtime, the actual network. This is the step most teams skip. It is the only step that matters.

Collect the failures. Each failure is a free design review. A missing dependency, an ambiguous exit code, a race between spawn and status — these are the edge cases your abstraction will need to handle. You now have them enumerated, not theorized.

Promote the proven logic. Once the script runs cleanly against reality, lift its logic into the tool as a native verb. Same logic, fewer keystrokes, callable from anywhere. The abstraction is not a redesign — it is the script with the rough edges filed off and the hard-coded paths replaced by configuration.

The Proof: A Missing Tool, Found in Five Seconds

During a session of building reconciliation logic for a five-agent deployment team, a script was written that read the live-state file, identified agents that had drifted from their declared configuration, and issued corrective spawn commands. The script was clean. The logic was correct. It had been reviewed by two people.

It was run on an actual host for the first time.

It failed immediately. A standard JSON parsing utility — present on every development machine, assumed to be universally available — was not installed on the production host. The script called it inline to extract a field from the live-state file. One missing package.

The fix took five seconds: swap the JSON call for a one-line shell alternative that extracts the same field using only POSIX tools. But the lesson was not about the fix. The lesson was about what would have happened if the reconciliation logic had been promoted to a native verb before that script ran.

The abstraction would have encoded the assumption. Every host in the fleet would have needed the utility installed. The dependency would have propagated silently into the tool's requirements, undocumented, until the next machine it ran on surfaced the same failure — but now buried inside a compiled binary instead of a readable script, and two abstraction layers away from the call site.

Because the script ran on the real thing, the assumption was visible. Because the assumption was visible, it was fixed. The native verb that was eventually promoted from that script has no external JSON dependency. It never did, because the prototype told the truth before the abstraction had a chance to lie.

What the Prototype Teaches That Diagrams Cannot

A diagram of a reconciliation loop has no failure modes. A running script has exactly the failure modes the environment produces. These are the only failure modes that matter.

The prototype also teaches the latency profile. A reconciliation loop that looks instantaneous on a whiteboard may take twelve seconds when it has to poll five agents over a mesh VPN and wait for their status responses. You will not know this until you run it. Twelve seconds changes the design: you parallelize the polls, or you cache the last known state, or you accept the latency and document it. Any of these is a valid choice. None of them appear in the design until the prototype makes the time visible.

The prototype teaches the output format. When you watch the script print its actions to a terminal, you immediately know whether the output is useful. Too verbose, and operators ignore it. Too terse, and failures are invisible. You tune this in minutes with a script. You submit a ticket and wait three weeks to tune it in a shipped feature.

When to Promote

The prototype is ready to promote when it has run successfully on real infrastructure more than once, when its failures are understood and handled, and when running it a second time produces no surprises. Not when the code is clean — it will never be clean. When the behavior is predictable.

Then lift the logic. Name the verb. Write the thin wrapper that calls the same sequence with the same guards and the same fallbacks, minus the scaffolding. The abstraction earns its existence by making a proven thing easier to call.

The rule: build the script before you build the feature, and run the script on the real system before you trust the script.

Agents in Parallel