Day 20

Architecture design and product thinking

You ship
An architecture document, a feature proposal and a 5-minute pitch.

What you'll build today

Over 19 days, WorkBot has grown a chat, a voice line, document intake, durable workflows, traces and evals. That is a lot of system for someone else to pick up, and no single document explains how it fits together. Today is the midpoint of the course, so you stop adding features and explain what you have.

First you document WorkBot as it is: what runs, where the state lives, which model does what, and why each choice beat its alternative. Then you take one of 3 startup briefs and design a system for it from the parts you already know. You learn to choose an architecture pattern for a job, and to put the guarantees in code rather than in a prompt. You also learn to tie each technical capability to a business outcome.

At the end of the day you have WorkBot's architecture document, and a design with a feature plan and a 5-minute pitch that a founder could act on. Day 40 reads both documents again.

By the end of the day you can

  • Design scalable AI system architectures for real startup problems
  • Make technical decisions with a clear analysis of their tradeoffs
  • Translate business goals into AI feature proposals, and pitch them

Topics

Architecture Patterns for AI Systems

  • Monolithic agent: a single agent with many tools. It is simple and works for most use cases. It fails when the tasks are too diverse.
  • Microagent architecture: specialized agents behind an orchestrator. It separates concerns better, and each agent scales on its own.
  • Event-driven: agents triggered by events, such as user actions, schedules and webhooks. It suits reactive systems and background processing.
  • Hybrid: different patterns for different parts of the system. Most real systems are hybrid.
  • Decision framework: the complexity of the task, the team's size, the scale required, the latency required

Designing for Reliability

  • Fallback strategies: model fallbacks from Opus to Sonnet to Haiku, provider fallbacks, and cached responses as a fallback
  • Circuit breakers: detect when an outside service is failing, stop sending it requests, and retry later
  • Graceful degradation: when an AI feature fails, the user sees a meaningful fallback
  • State management at scale: workflows, checkpoints and runs that resume, built on Day 18's Inngest patterns
  • Idempotency: making sure an agent's actions can be retried safely, without repeating their side effects

Product Thinking for AI Engineers

  • The framework: business goal, then user pain points, then AI feature proposals, then a feasibility assessment
  • What AI can do reliably today: classification, extraction, summarization, search, generation, conversation
  • What AI struggles with: precise math, real-time data, guaranteed correctness, complex multi-step reasoning with 100% reliability
  • Feature prioritization: an impact against effort matrix for AI features
  • Managing founder expectations: what to promise, what to caveat, and how to scope a minimum viable product (MVP)
  • Open-source software (OSS) or SaaS: build or buy for each component, and the criteria for choosing

Build

Features shipped today

  • WorkBot: The current system written up in the architecture document, and a design against Brief A, B or C, pitched

The Midpoint Challenge: Given a real startup brief, design a complete AI system and pitch it.

Startup brief (choose one, or be assigned one)

Brief A: AI Customer Support Platform

An e-commerce company with 10,000 daily support tickets wants to automate 80% of responses while maintaining quality. They have a product catalog, order system, and knowledge base.

Brief B: AI Legal Document Analyzer

A law firm wants a tool that reads contracts, identifies risks, extracts key terms, and generates summary reports. They process 200+ contracts per month.

Brief C: AI Sales Intelligence Tool

A business-to-business (B2B) SaaS company wants an AI that monitors prospect companies, identifies buying signals, drafts personalized outreach, and tracks engagement.

Deliverables , as the 3 sections of one design document

  1. Architecture document (2-3 pages):
    • System diagram showing all components (agents, data stores, APIs, UI)
    • Agent design: which agents, with which tools and patterns
    • Data architecture: what is stored where, and how data flows
    • Technology choices: the tools and services from the course for each component
    • Tradeoff analysis: why these choices over the alternatives
  2. Feature proposal (1 page):
    • 5 AI features prioritized by impact against effort
    • MVP scope: what to build first
    • What to promise the founder, and what to caveat
  3. Pitch presentation (5 minutes):
    • Present the architecture and the feature proposal as if pitching to the startup's founder
    • Show that you understand their business problem
    • Map technical capabilities to business outcomes
    • Handle questions about feasibility, timeline and cost

This is the midpoint design exercise. Everything from the first 4 weeks should inform the design. Day 40 revisits it with WorkBot as built.

Step by step

Document WorkBot as built

Start with the system you know best, because the patterns you name in it are the ones you reuse for the brief. The document runs 2 to 3 pages with one diagram, the length the curriculum gives an architecture document. Write it for the engineer who will change the system without you. Say what runs, where the state is, which model does what, and why you made each choice over the alternative.

Study first

  • The patterns in today's topics. WorkBot's assistant is one agent with tools, which the topics call a monolithic agent, and Day 15's research analyst is specialists behind an orchestrator. The document approval and the schedules from Days 18 and 19 are event-driven workflows. A hybrid is a system that uses more than one of these, and WorkBot is one, so your document names which part uses which pattern, and why.
  • A tradeoff names the alternative and its price, in the form "X over Y, because A, at the price of B".
  • Anthropic's "Building effective agents". Start with the simplest solution, a single call or a workflow, and add an agent's autonomy only where the task needs it. Use a workflow when you know the steps in advance.

Build

  1. Create docs/architecture.md and draw the system diagram in Mermaid, a text format for diagrams that GitHub renders in place. Show the entry points (the web app, the terminal and the API), the app, the workflow engine, the database, the models, and what WorkBot calls outside.
  2. Add a table of the agents, with each one's pattern, model, tools and what ends a run. Below it, add a paragraph on why each pattern has the fewest moving parts that handle its task.
  3. Describe the data, what is stored where and why, in 5 lines. Then trace the 2 flows that matter, a chat message and a document.
  4. List the technology as a table of need, choice and reason, and the tradeoffs as a list, each with the alternative and its price.
  5. Describe reliability as built and as missing. Name the limits, retries, fallbacks and guards that exist. Then name the fallback chain, circuit breaker, response cache, shared rate limit, transcripts and sign-in, which do not exist yet.

Done when

  • You give the document, and nothing else, to a colleague or to an agent, and ask 4 questions. Where does a chat message's memory come from? What happens to a document whose extraction is doubtful? Which model answers which call, and what does it cost? Which component has no fallback when its service is down?
  • Each answer comes from the text.
The system diagram
WorkBot after Day 19, as the architecture document draws it, with the entry points, the app, the workflow engine, one database, the models and the outside services.
Technical notes
  • 4 files are the sources for the document's claims, so a reviewer reads them beside it. The step limits are in src/lib/agent.ts, the tables in src/lib/schema.ts, the tools in src/lib/tools.ts, and the prices in src/lib/providers.ts.
  • List the missing capabilities in the document as proposed backlog items.
  • The diagram is the Mermaid block in the document, which Chrome rendered to the image above. The document, not the image, is the source.

Design a system for a brief

With WorkBot written down, you design for someone else's problem, with the same patterns and parts. The curriculum offers 3 briefs, so choose one. The reference run took Brief A, and the rest of the day uses its design as the example. In Brief A, an e-commerce company with 10,000 daily support tickets wants 80% of responses automated with quality held. The reference run's design defines that quality as a judge's mean of 0.9 per kind of ticket, and fewer than 1 wrong reply in 100 on audit. Reuse WorkBot's components.

Study first

  • The decision framework's 4 criteria decide the shape. For Brief A, the tickets are short, so the task needs one agent with tools, and the team is 2 engineers, so the design keeps few moving parts. The scale is 10,000 tickets a day, so it needs a queue and a concurrency limit. An automatic reply should take a minute, so the small model goes first. Google Cloud's guide to choosing a design pattern for an agentic system asks about the task, latency, cost and human involvement. It also starts from a single agent.
  • What AI does reliably today, in the topics' list: classification, extraction, summarization, search, generation and conversation. It struggles with precise math, real-time data, guaranteed correctness, and long chains of reasoning at 100% reliability. So the reference run's design puts the guaranteed parts in code, with rules for refunds, an approval over a limit, and a confidence bar per kind of ticket. The bar is the score below which an answer goes to a person instead of out to the customer.
  • Build or buy, per component. The reference run's design keeps the help desk, buys the workflow engine and the models, and owns the data and the evals. Martin Fowler's utility versus strategic dichotomy gives the rule behind it. Buy what does not set the business apart, and build what does.

Build

  1. Write your design as one document in docs/design/, which in the reference run was docs/design/brief-a.md. Section 1 is the architecture. Give it the workflows, a diagram, and a table of the agents with model, tools and limits. Add the data and why it lives where, the technology, the decision framework and the build-or-buy calls, and the tradeoffs.
  2. For Brief A, the reference run's section 1 had 3 loops as 3 workflows, resolve, escalate and learn, and 4 agents, triage, resolver, actor and judge. Its technology was WorkBot's stack, with the help desk kept as the channel.

Done when

  • Section 1 of your design names the workflows, the agents with their model, tools and limits, where the data lives and why, and the tradeoffs.
The 3 loops of the support platform
A ticket's path through the platform the reference run designed against Brief A, in 3 loops called resolve, escalate and learn.
Technical notes
  • In the reference run's design, actions are a second agent with its own tools behind policy rules, so nothing in the resolver's prompt can refund an order. The model proposes a refund, and code checks the policy and asks a person above the limit before anything moves.
  • The help desk stays, because the human support agents' history and habits are there, and because a rollout per kind can be paused there.

Plan the features and the pitch

A founder decides on a plan, not a diagram, so the rest of the design turns the architecture into features, numbers and a pitch. State the build order, the timeline and the cost in the founder's numbers.

Study first

  • The product-thinking framework in today's topics runs from business goal to user pain, AI feature and feasibility. For Brief A, the goal is the founder's, 80% automated with quality held. The pain belongs to the human support agents, who answer a few kinds of question all day, and to the customers, who wait for a reply. Feasibility is what the eval set says, and in this design the eval set is golden cases like Day 19's, built from the support agents' own edits. The chapter on user needs and defining success in Google's People + AI Guidebook asks whether AI adds value that a simple rule could not.
  • Impact against effort. Order the features in a 5-row table, and draw the minimum viable product (MVP) around the first 3. Nielsen Norman Group's guide to prioritization matrices shows how a team places items on 2 axes, such as value and effort.

Build

  1. Write section 2, the feature proposal. List 5 features by impact and effort, with the week each lands, and make the first 3 the MVP. Say what to promise, which is the things you can measure, and what to caveat. For Brief A, the reference run's caveats were that the rate ramps up kind by kind and the model is wrong sometimes. It also said that refunds need rules, and that the knowledge base needs an owner.
  2. Write section 3, the pitch, in 6 parts with their seconds. Open with the problem in the founder's numbers, and tie each technical capability to a business outcome. In the reference run, triage tied to routing time, drafts to agent time, and the confidence bar to the error rate. Put the ask last, and list the questions to expect with their answers.
  3. Give the numbers with their assumptions. For Brief A, the reference run assumed an average of 7 tickets a minute over 24 hours, with peaks of 3 to 5 times that. A ticket of a common kind takes 3 calls of 2,000 input and 300 output tokens on the small model. The 20% that triage marks as complaints or multi-issue take 4 calls of 3,000 and 500 on the large model. That is about $96 a day in model cost at 10,000 tickets, budgeted at $250.

Done when

  • A founder who reads the 3 sections can say what they get in week 3 and what it costs a day. They can also say what the system will refuse to do alone, and what they must provide.
Technical notes
  • Treat the automation rate as an outcome, not a setting. In the reference run's design, each kind of ticket has a confidence bar, raised as the kind's eval score earns it, and nothing sends in week 1. Promising 80% on day one would mean sending replies the judge would fail.
  • Cost per ticket comes from the prices in src/lib/providers.ts. The small model costs $0.10 per million input tokens and $0.50 per million output, so 3 calls of 2,000 in and 300 out come to $0.00105. The large model costs $2 and $10 per million, so 4 calls of 3,000 in and 500 out come to $0.044. At 80% common tickets and 20% complaints, that averages about $0.0096 a ticket.
  • The judge's 10% sample, retries and cache writes are the margin between $96 and the $250 budget.

Resources