Agent fleets: 50 concurrent agents, cloud agents and test automation
You ship
A 50-task fleet run measured on merges per hour, reverts and brief-to-merge time.
What you'll build today
On Day 21 you shipped 3 tasks, one agent at a time, and watched each one work. A real backlog is much longer than that, and while you watch one agent per task, your own attention sets the pace.
Today you run a fleet, many coding agents working at once. Each of 50 small tasks gets its own agent, some in worktrees on your laptop and some in cloud sessions. You learn that the limit moves away from the agents. First it is the quality of the briefs, so you specify every task, with the test to write first, before any agent starts. Then it is review, so 4 reviewers, CI and a queue that merges in batches check every pull request before it merges.
By the end of the day the fleet has worked through the backlog. You have the numbers to judge it by: merges per hour, reverts, and the time from brief to merge.
By the end of the day you can
Run dozens of coding agents at once on well-specified tasks, and merge the results safely
Use cloud agents for work that outlives a laptop session
Automate testing and review, so you can trust a fleet's output at merge time
Topics
From 4 Agents to 50
What changes: the quality of the task specifications becomes the bottleneck, then review. You become the dispatcher and the reviewer
Task shape: small, independent and verifiable, with an acceptance test written before the agent starts
Isolation: a worktree or a cloud sandbox for each agent, and no shared mutable state
Dispatch: a backlog of specified tasks, a dispatcher that assigns them to local worktrees and cloud sessions, and a dashboard of their state
Cloud Agents
Platforms: Codex cloud tasks, Claude Code on the web, GitHub's coding agent. A task goes in, a pull request comes out, and each run gets its own environment
Environment definitions: the dependencies, secrets and test commands the cloud agent needs, with the same definition used locally
Cloud or local: cloud for long tasks, many tasks and clean machines, and local for tight iteration with a person
Cost and limits per platform, and queueing when you hit a limit
Test Automation for a Fleet
Every pull request: the full test suite, type-check, lint, and the eval smoke subset, a few of the Day 19 evals (scored tests of WorkBot's output). Fix or quarantine a flaky test the day it appears
Generated tests: the agent writes tests from the acceptance criteria before it implements. The reviewer checks the tests first
Property-based and snapshot tests for generated code, and contract tests between monorepo packages
End-to-end tests against a preview deployment for each pull request (Day 1)
Pull Request Review at Scale
Review fleet: the 3 Day 2 reviewers (bugs, performance, security) plus a brief reviewer, which checks the change against the brief's criteria. They review every pull request before a person looks
Human review: the brief, the tests, the diff. Approve, request changes, or close with a better brief
Merge queues: main stays green when 50 pull requests land in an afternoon
Metrics: pull requests merged per hour, revert rate, time from brief to merge
Build
Features shipped today
WorkBot: Tasks from its backlog, shipped by the fleet
Run a fleet against your backlog:
Specify 50 small tasks from your backlog, each with acceptance criteria and a test. Review the specifications before any agent starts
Dispatch: 20 to local worktrees and 30 to cloud agents, with a dashboard that shows each task's state
Automation on every pull request: the review fleet, the test suite, the eval smoke subset and a merge queue
Merge: review the pull requests as they complete. Record the number merged, the reverts and the time from brief to merge
Retrospective: which task shapes succeeded, which failed, and what the next backlog should look like
Step by step
1
Fifty tasks that fit an agent
On Day 4, with 4 agents, the day's limit was the model's output. With 50 agents, the limit is the brief, so the day starts with the tasks. A good task is one an agent can finish in one session, that no other task collides with, and whose outcome a test can check.
Study first
Task shape: a task has one goal, criteria an agent checks one by one, constraints that say what must not change, the files involved, and the test to write first. When 2 tasks name the same files, they become 2 pull requests that conflict. Claude Code's best practices explain the test first, in giving Claude a way to verify its work. Without a check it can run, an agent stops when the work merely looks done.
Where tasks come from: one source is a survey of the code by area, such as libraries and routes, CLIs and docs, or pages and components. Each gap the survey finds cites the file and the behavior. The other source is the previous day's review findings, which are briefs already.
Review the specifications before any agent starts, with a second model as the reader. It looks for pairs of tasks that change the same file, wrong claims about the code, criteria an agent could not check, and tasks too large for an hour.
Build
1
Write docs/fleet-tasks.md with 50 ## sections. Each section has Goal, Acceptance criteria, Constraints, Related code and docs, and Test, the same fields as the issue form from Day 21.
2
Survey the code in 3 areas, for 25 candidates each. The first area is libraries, routes and workflows, the second is CLIs, docs, the site and the eval sets, and the third is pages and components. Every candidate names a file, a gap and a test. Keep 50 that are worth a session, no two of which change the same file, and swap in the previous day's review findings, which are briefs already.
3
Have another model review the specifications for pairs that change the same file, wrong claims, criteria an agent cannot check, and tasks that are too large.
4
Add a file command to scripts/fleet.mjs, the script that runs the fleet. node scripts/fleet.mjs file docs/fleet-tasks.md then files one issue per task, labeled fleet.
Done when
50 issues exist, no two change the same file, and every one names the test to write first.
One of the 50 tasks, with its goal, the criteria an agent checks one by one, the constraints, the files and the test to write first.Technical notes
In the reference run, the survey was 3 read-only agents, one per area. Each was told to cite line numbers, and to group candidates so that no 2 touch the same file. The 75 candidates became 50 after the review findings from Day 21 took 4 slots.
Tasks are independent when they change different files. Reading is fine, so a task may read src/lib/requests.ts while another changes it.
In the reference run, the specification review found no overlaps and no wrong claims. It made 10 tests more precise by naming an assertion for each criterion that had none.
2
Dispatch to worktrees and the cloud
With 50 issues filed, each one needs an agent and a working tree of its own. On your laptop, that is a git worktree with its own dependencies, worked by a non-interactive Claude Code session. In the cloud, it is a session that clones the branch, does the work, and pushes.
Study first
Worktrees: create one with git worktree add -B fleet/<issue> ../workbot-fleet/<issue> day-22, then run pnpm install --frozen-lockfile in it. The session is claude -p "<the brief>" --model sonnet --session-id <uuid> --permission-mode acceptEdits --allowedTools <the list>, where the list allows edits, reads, pnpm, git, gh pr create, gh pr view and gh issue view. Add --strict-mcp-config with an empty config to keep the Model Context Protocol (MCP) servers out.
Cloud sessions: claude --cloud "<the brief>" --model sonnet creates a cloud session that clones the repository's GitHub remote at the current branch, so push first. The command wants a terminal, so the dispatcher, your script scripts/fleet.mjs, runs it under script -q /dev/null (script -q -c … /dev/null on Linux). The session can push a branch, but it holds no GitHub token, so the dispatcher opens the pull request when the branch appears.
Concurrency: there are 2 lanes, local and cloud, each with a number of slots, and the dispatcher refills the slots every round, one pass of its loop. The limits are the laptop and your Claude plan's rate limits, not the script.
Build
1
Add scripts/fleet.mjs dispatch --local 6 --cloud 12. It fills the local slots first, then the cloud ones, and when a start fails it logs the failure and retries in the next round.
2
Keep the fleet's state in the issues and pull requests on GitHub, plus .fleet.json at the repository root, which git ignores. That file has one entry per task with its session's process id or its cloud id, and later the head commit the queue reviewed.
3
Write the brief each agent gets: the issue number, the branch, "write the test first and see it fail", the checks to run, and one commit with Closes #n. A local agent's brief also asks for the pull request body in the template's shape, with its session id.
4
Add scripts/fleet.mjs status, which prints one row per task with its state: queued, local or cloud with the minutes running, a pull request with its CI state, merged, or failed.
Done when
Every task is running or has a pull request, and the board shows each task's lane and how long it has run.
The dashboard a few rounds in, with local and cloud agents, pull requests waiting on CI, and merged tasks.Technical notes
claude -p prints only its final message, so a local agent's log stays empty until it stops. Watch the branch and the pull request for progress instead.
claude --cloud refuses a pipe for stdout, and script -q /dev/null claude --cloud … provides the terminal it asks for. The first run shows the workspace trust dialog, once.
In the reference run, the cloud environment had no setup script, so the brief asks for pnpm install first. A cloud session's log is on its page at claude.ai.
A local session holds the UUID passed with --session-id, which goes into the pull request's Session section. A cloud session puts its claude.ai link there instead.
3
Review and merge in batches
Once the pull requests come back, the merge step decides whether to trust each change. Every pull request meets CI, 4 reviewers and a queue, which merges in batches and checks the branch after each batch.
Study first
The review fleet: the 3 reviewers from Day 2 (bugs, performance, security) plus a brief reviewer. The brief reviewer reads the issue's criteria, then the tests, then the code, and reports each criterion as met and tested, met but untested, or not met. The skill ends with one line that a script can read, Verdict: approve or Verdict: request changes.
The merge queue: GitHub's merge queue works only for repositories an organization owns, not for a personal one, so the fleet script has its own. A batch is the ready pull requests of one round, up to 6. The queue reviews them in parallel and merges the approved ones one at a time, with the agent's commit message. Then it runs one full check on the branch, and the eval smoke subset. An eval is a scored test of WorkBot's output (Day 19), and the smoke subset is the 5 answer cases of the Day 19 golden set, run with pnpm eval --only answer.
The revert loop: if the check fails, the queue reverts the last merge by its commit and runs the check again, until it passes. It reopens each reverted issue with the failing lines.
Fix or quarantine a flaky test the day it appears, because a queue amplifies it.
Build
1
Write .claude/agents/brief-reviewer.md. Then change your Day 2 review skill, /pr-review, so that it runs all 4 reviewers and ends with the verdict line.
2
Add scripts/fleet.mjs merge --batch 6. It picks the ready pull requests, which are open, have passed CI, and have a head commit the queue has not reviewed, and it records that head in .fleet.json. It posts each review as a comment and squash-merges the approved ones with the commit's subject and body. Then it runs pnpm check and pnpm eval --only answer on the branch, and the revert loop.
3
When a review requests changes, or a check fails, start one more session on the pull request's branch with the review as its brief. If a second verdict goes against it, a person decides.
4
Add scripts/fleet.mjs run, which runs dispatch, merge and status every 2 minutes until every task is merged, and then prints the report.
Done when
Every merged pull request carries a review comment with a verdict.
The branch is green after every batch, reverts included.
What the queue cannot decide stays open for a person, such as a blocking item outside the brief, or a criterion the brief got wrong.
The 4 reviewers' findings merged into one comment, the brief's criteria checked one by one, and the verdict line the queue reads.Technical notes
Reviews run as claude -p "/pr-review <n>" with the agent tool allowed, so the skill fans out to its 4 reviewers as it does in an interactive session.
A squash merge keeps one commit per task, with the agent's message. Closes #n then closes the issue, and the history reads as 50 small changes.
The check after a batch is the full pnpm check, which takes 3 minutes, and that is why the queue checks once per batch rather than once per pull request. A check can also fail for reasons outside the batch. In the reference run, one failed under the load of 6 fix sessions and reverted an innocent change, which passed on its own and was reinstated. Keep the log of every check for that reason.
A smoke eval guards against breakage, not against drift, a slow loss of quality. In the reference run, 12 runs of the 5 cases on the small model scored between 0.60 and 0.80, on batches that touched no prompt. The bar is 0.5 because the gate is there to catch an assistant that stopped answering, and a change to a prompt gets the full eval from Day 19.
4
What the numbers say
When the last task has merged, the fleet's numbers tell you whether the day worked, and the task shapes tell you what to change in the next backlog.
Study first
The metrics that matter for a fleet: pull requests merged per hour measure throughput, and the revert rate measures the quality of the specifications and the review. The time from brief to merge, from the issue's creation to the merge, measures how long the queue holds work. The number merged says the least, because 50 merged says nothing about whether those were the right 50. These mirror the software delivery metrics from DevOps Research and Assessment, where change lead time and change fail rate play the same parts.
The retrospective: write down which task shapes succeeded, which failed, and what the next backlog should look like.
Build
1
Add scripts/fleet.mjs report, which prints merges per hour, reverts, and the median and 90th percentile (p90) of the time from brief to merge.
2
Write docs/fleet.md with the numbers, the task shapes that worked and the ones that did not, with the issue for each, and what the next 50 tasks should look like.
Done when
The report prints, and docs/fleet.md names the task shapes, not just the numbers.
What the fleet did, in merges per hour, reverts and the time from brief to merge.Technical notes
In the reference run, 50 tasks merged over 2 hours 26 minutes from the first merge to the last, 20.6 an hour. Of those, 25 came from worktrees and 25 from cloud sessions. The queue merged 44 on an approving review, and a person merged 6 after reading the review, each with the reason on the pull request. There were 52 reviews, and 17 second sessions ran after a review or a failed check, of which the queue then merged 11. CI ran 63 times and passed 59, and the one revert was reinstated. The median time from brief to merge was 101 minutes and the 90th percentile 160, over 15 rounds of the loop. The test count went from 166 to 298.
The tasks below are cited by their numbers in docs/fleet-tasks.md, F-01 to F-50. The shapes that merged on the first review were single-module fixes with a named test.
Three shapes needed a second session. One was a fix that introduced its own problem, when F-02 added a trim that runs in quadratic time on a hostname that request input reaches. One was a brief that asked for a heuristic where the problem needed a mechanism. F-05 was about a pattern that can hang the server, and the second session bounded the match with a timeout. The last was a test that depended on the clock, because F-15's test used rows inserted in the same millisecond.
A non-interactive agent cannot finish one shape: F-33 edits .claude/settings.json, the agent's own permissions. Another shape fails quietly, when a cloud agent pushes to the branch its environment named instead of the one the brief named.
In the reference run, a person specified and reviewed the 50 tasks before any agent started. That person finished one task by hand, read and decided 7 reviews, undid one revert, and stopped the loop when what was left was judgment. It took about 2 hours over the afternoon, most of it spent reading reviews.
