Day 29

Config-first design, grammars and meta-compilers

You ship
WorkBot config-first, with a second tenant from configuration, a rule language with 20 tests, an entity generator, and its check as a cached task graph.

What you'll build today

After Day 28, WorkBot has a data layer with one shape, rules and a cache, and its customer lives in one file. Much of what a customer would want to change still sits in code, though. The routes, the assistant's name on each page and the rules are code or plain JSON, and every new kind of record takes hand-written files.

Today you make configuration the behavior and code its interpreter. You put configuration in levels with one precedence, and you bring up a second tenant, a second customer with its own settings, from a file alone. You turn the rules into a small language with a grammar, so a model can propose a rule that a person reads and approves. You generate whole entities from short descriptions, and you run the check as a cached task graph.

By the end of the day, WorkBot serves 2 tenants from configuration, with a rule language, an entity generator and a check that skips unchanged work. The monorepo of apps and packages waits, written down, until a second app exists.

By the end of the day you can

  • Design products where behavior is configuration and code is an interpreter
  • Write a small grammar in Backus-Naur form (BNF) and a parser for a domain language, and generate code or configuration from it
  • Structure a monorepo so agents and humans can navigate and change it quickly

Topics

Config-First Code Design

  • The split: schemas (what exists), configuration (what this deployment does), and code (how configuration is interpreted)
  • Zod schemas type the configuration, and WorkBot validates it at startup
  • Levels: product defaults, tenant overrides and user preferences, with precedence rules written once. A tenant is how WorkBot keeps one customer's data and settings apart
  • Configuration as the agent's workspace. The agent proposes configuration changes, tests run, and a human approves. Code changes are rarer and reviewed harder
  • Anti-patterns: configuration that is really code (nested conditionals in YAML), and code that is really configuration (hard-coded prompts and routes)

Domain Languages + BNF Grammars

  • When configuration is not enough: rules, workflows, templates, query languages. A small language with a grammar is easier to test than a JSON tree of conditions
  • BNF and extended BNF (EBNF): terminals, non-terminals and productions. Reading a grammar, and writing one for a rule language
  • Parsing in TypeScript: parsing expression grammar (PEG) parsers such as Ohm, and parser toolkits such as Chevrotain. The parser turns text into a parse tree, then an abstract syntax tree (AST), which an interpreter runs
  • Grammars and models. Constrained decoding and JSON Schema structured outputs are grammars enforced while the model generates. One grammar validates model output and human input

Meta-Compilers + Meta Design

  • Meta-compiler: a description of a language goes in, and a parser or interpreter comes out
  • The same idea at product scale: a description of a product (entities, rules, screens, tools) that generates the product
  • Generators already in use: object-relational mappers (ORMs) from schemas, API client code from OpenAPI, forms from Zod, and UI from json-render catalogs (Day 9)
  • Designing for generation: keep the description small and declarative, and let the irregularity live in the generator
  • Failure modes: the description grows its own conditionals, or someone edits generated code by hand

Monorepo Structure

  • pnpm workspaces + Turborepo: apps (web, api, workers), packages (schemas, config, agent, integrations, ui), and tooling
  • One lockfile, one type-check and one CI. Task graphs and caching let the whole repository build in minutes
  • Conventions agents can follow: where things go, how packages import each other, and an instruction file per package
  • Shared types end to end: a schema package imported by the API, the workers and the UI

Build

Features shipped today

  • WorkBot: Per-customer configuration, the rule language and the entity generator. The routes, the age of a cached page and each customer's rules move to configuration, with a second tenant from a file. The check runs as a cached task graph, and the move into a monorepo of shared packages is written down for when a second app exists

Restructure WorkBot into a config-first monorepo:

  1. Monorepo with pnpm workspaces and Turborepo: apps/web, apps/api, apps/workers, packages/schemas, packages/config, packages/agent and packages/integrations. Shared types, and one CI pipeline with caching
  2. Config-first: tools, prompts, routes, provider routing (Day 24) and tenant overrides as Zod-validated configuration loaded at startup. Bring up a second tenant with configuration only
  3. A rule language: the BNF for the Day 28 rule table (conditions, actions, ordering), a parser with Ohm or Chevrotain, an interpreter, and 20 tests. The agent proposes rules in the language, tests run, and a human approves
  4. A generator. From a declarative entity description (fields, validations, permissions), generate the Postgres migration, the Zod schema, the API route and the form. Add a second entity with only the description
  5. 3 end-to-end tests that stay in CI: a new tenant onboarded from configuration, a rule authored by the agent and approved, and an entity generated end to end

The steps below keep one deployable, the decision of Day 26, so they do not split WorkBot into apps and packages. They run the check as a Turborepo task graph over the app and the site. They write the apps and packages layout down as the move to make when a second app, or a second consumer of a package, exists.

Step by step

Configuration in levels

The first move puts every setting a customer might change into configuration, read in one order of precedence. A second tenant then needs a file and no code.

Study first

  • Schemas say what exists, configuration says what this deployment does, and code says how to interpret the configuration. A prompt with the company's name in it is code that is really configuration, and a YAML file with nested conditionals is configuration that is really code. The Twelve-Factor App's chapter on config draws the line at what varies between deploys. It keeps those values in environment variables, while WorkBot keeps a customer's values in a file that one variable, CUSTOMER, names.
  • Configuration comes in levels: the product's defaults, a tenant's overrides, the environment and a user's preferences. A tenant is how WorkBot keeps one customer's data and settings apart, and you write the precedence between the levels once.
  • WorkBot parses every level at startup, so a wrong file or variable fails at startup, naming the field or the pair, rather than at the first request. The rules file is parsed at the same startup, but it is not a level, because it has one source.
  • The browser cannot read a file chosen at startup. So the server reads the configuration, and a client component gets what it needs as props. Next.js shows the pattern in passing data from server to client components, and the props must be serializable. A schema that depended on the customer becomes a function of what the server passes.

Build

  1. Write one content/customers/<id>.json per customer, with settings for what the customer overrides. Write content/defaults.json, which the product starts from, for the routes and how long a cached page is kept. Add CUSTOMER to src/lib/config.ts, naming the file a deployment serves.
  2. Write src/lib/settings.ts, with the Settings and Overrides schemas and resolve(defaults, overrides, env), which holds the precedence in one place. A customer's value replaces the default's, field by field, and a route from ROUTES goes first among the models its task tries. src/lib/routing.ts reads the resolved routes and nothing else.
  3. Remove the import of the customer from src/lib/request-model.ts. There, requestModel(teams) builds the schemas for a list of teams, and RequestDraft is what a client component composes. src/lib/requests.ts builds the real schemas from the customer on the server, and the catalog takes the teams the same way.
  4. Bring up a second tenant from a file, content/customers/harbor-dental.json, with its own teams and assistant, and override the chat's models in settings.routes.chat. The assistant's name reaches every page from the file, and the chat, a client component, takes the name as a prop. In the reference run, 12 places had said WorkBot in code.
  5. Write tests for the precedence, level by level, and for a bad route or task refused at load. Add a run under CUSTOMER=harbor-dental that checks the name, the teams, the prompt and the overridden route in one go.

Done when

  • CUSTOMER=harbor-dental pnpm build && pnpm start serves Harbor Dental's assistant with no change to code.
A second tenant
Harbor Dental's assistant from a file, with its name, its teams and its chat route. The code is unchanged.
Technical notes
  • In the reference run, the refactor touched 14 files, nearly all of them import lines. Modules that read teams from the model now read it from requests.ts, which builds the model from the customer.
  • One action changed shape. The catalog's submit_request keeps its team check only when it is built with the teams. The browser's registry never prompts a model, so it builds the action without them, and the API checks the team.
  • The trigger paths of the eval workflow name content/customers/**, content/defaults.json and src/lib/settings.ts, because a prompt is a function of the customer and a route decides the model.
  • CUSTOMER is read when the app starts, and a page prerendered at build carries the build's customer, so a tenant is a build. Serving the second tenant from the first tenant's build shows the first tenant's brand on screen, as it did in the reference run.
  • Bringing up a tenant can show routing rules that name teams it lacks, and the app then refuses to start, as Harbor Dental showed in the reference run. So split the rules. The product's rules, which name no team, go in content/rules.rules, and a customer's rules go beside its file, read after the product's.
  • User preferences, the third level in the curriculum, are not built, because WorkBot has no users yet.

A language for the rules

The rules of Day 28 sit in a JSON file, which is hard to read in a diff and hard for a model to write. A small language with a grammar fixes both, and it gives the next part a format the model can propose in.

Study first

  • A JSON tree of conditions is hard to read in a diff and hard for a model to generate. A small language with a grammar reads well in a diff and asks less structure of a model, and one grammar validates a person's line and the model's.
  • A grammar in Backus-Naur form (BNF), or extended BNF (EBNF), is built from terminals, non-terminals and productions. Learn the difference between a lexical rule and a syntactic one, which skips whitespace, and keep keywords out of identifiers.
  • Ohm builds a parsing expression grammar (PEG) parser in TypeScript from a grammar. Its semantics turn the parse tree into the objects the program already uses. Constrained decoding applies the same kind of grammar while the model generates, and WorkBot applies it after generation, in the parser.

Build

  1. Write the BNF in docs/config.md, and the Ohm grammar in src/lib/rule-language.ts. Make the keywords lexical rules, and define ident as a name that is not a keyword.
  2. Write semantics that build the rule objects of Day 28, so the engine does not change. parseRules fails with the line and column, and printRules writes a table back as the language.
  3. Replace the JSON with 2 files. The product's 3 rules go in content/rules.rules, and the customer's routing rules go beside its file in content/customers, read after the product's. The grammar allows comments.
  4. Write 20 tests that cover every production, the escapes, and the errors with their positions. They also cover keywords refused as names, the empty text, the round trip, and the file itself.

Done when

  • The Day 28 table, printed and parsed, is the same table.
  • A line with a typo fails at startup with its line and column.
Technical notes
  • In a quoted string, \" and \\ decode to a quote and a backslash, and any other escape keeps its backslash, so "\d" is the pattern \d. The printer escapes quotes and backslashes, and what it prints parses back to the same rule.
  • Ohm checks the arity of a semantic action against the production, so an action for normalize "phone" takes 2 arguments even though it uses none.
  • The Day 28 schema still validates the parsed objects, so ids must be unique and routed teams must be real.

Rules the model proposes

Once the rules are readable, a model can write them. The model proposes a rule, the parser and a dry run check it, and a person approves it, which makes configuration a place where the agent can do safe work.

Study first

  • Configuration can be the agent's workspace. The agent proposes a change to a file, a check runs, and a person approves. The language makes the proposal readable, and the parser lets WorkBot refuse a bad one. Anthropic's guide to building effective agents describes the loop, where feedback on an attempt goes back to the model. It also describes agents that pause at checkpoints for a person's judgment.

Build

  1. Write src/lib/rule-proposal.ts, where proposeRule asks the model for one line and sends it the grammar, the fields of each kind, the customer's teams and the rules so far. A line that does not parse, repeats an id or names a team the customer lacks goes back to the model once, with the error.
  2. In the same file, dryRun applies the rule alone to up to 100 stored records of its kind, rows the rules already shaped, and lists what would change. approve appends the printed line to the customer's file, since a proposal belongs to the customer.
  3. Add pnpm rule "<what the rule should do>" [--approve] in cli/rule.ts.
  4. Write tests with a scripted model for the retry with the error, the refusal, the dry run on 2 tickets, and the file one line longer. Write a live test that asks the real model for a routing rule, and checks that the rule parses and routes to the right team.

Done when

  • A rule the model wrote is in the file, approved by a person after seeing what it would change.
The model writes a rule
pnpm rule takes one sentence and gives back one line in the language. It shows the 2 tickets the rule would change, then the line approved into the file.

Entities from descriptions

A new kind of record still takes a table, a module, a route, a test and a page, all written by hand. A generator turns a short description into all of them, the same idea as the parser, applied at product scale.

Study first

  • A meta-compiler takes a description of a language and gives back a parser, and at product scale a description of a product's parts gives back the parts. WorkBot already uses generators: Zod to JSON Schema for the tools, the catalog to UI (Day 9), and the grammar to the parser above. Zod's page shows the conversion, and the types it refuses because JSON Schema cannot express them.
  • To design for generation, keep the description small and declarative, put the irregularity in the generator, and never edit what it writes. Watch for 2 failure modes: the description grows its own conditionals, or someone edits generated code by hand. Prettier's API formats a generated string with the repository's own settings, through format and resolveConfig.

Build

  1. Write src/lib/entity-model.ts, a schema for a description with a name, plural, label, access, and fields. Each field has a type, and can be required or have a max, multiline, options or a default. The schema refuses what a generator cannot handle: no required field, names that repeat, or a choice's default outside its options.
  2. Write src/lib/entity-generator.ts, which turns one description into files, as template strings. It writes the table statement, the module with the Zod schemas and the queries, and the route and its types, shaped like the hand-written ones. It also writes the page with its form and its action.
  3. Have the generator also write a test that creates a record from the required fields, and is refused without the first one. Its index writes the 2 files the hand-written code imports, the tables for the schema and the pages for the navigation.
  4. Write cli/generate.ts. pnpm generate writes the files, formatted with the repository's Prettier, and removes generated files that no description asks for. pnpm generate --check fails when a file differs from its description or has none, and it runs first in pnpm check.
  5. Write content/entities/vendor.json, then location.json, as descriptions alone. pnpm generate writes their files, and their generated tests run with the rest.

Done when

  • A second entity exists with only a description written.
  • Its route creates and refuses as the generated test says.
  • A hand edit to a generated file fails the check.
An entity from a description
/admin/vendors, generated from a 19-line description. The form comes from the fields and the table from the rows, with a vendor added through the form.
Technical notes
  • In the reference run, the description of locations is 24 lines, and it becomes 253 lines in 6 files. The Day 26 audit counted 10 files touched for a new request field, but a field on a generated entity touches 1.
  • Keep the table statements out of the entity's module. That module imports the database, which imports the schema, and if the schema imports the module back, the statement is read before it exists and the table is never made. In the reference run, the generated route test was the first to show this, and the statements now live in a file that imports nothing.
  • An entity's access decides where its files go. An employee entity's page goes under /admin and its API under /api/admin, while a customer entity's files go at the root, so one check on the admin prefixes covers both. Nothing in the app is authenticated yet, and Day 30 adds the check.
  • Put an employee entity's API under the admin prefix. In the reference run, the first version put every API at /api/<plural>, outside the prefix, and the review caught it. pnpm generate now removes the files a description no longer asks for, so the move left nothing behind.
  • The form does not send a field left empty, so a default applies. The API refuses what the form cannot, and names the field.

The check as a task graph

With more generated and configured parts, pnpm check does more work on every change. A task graph runs the tasks in parallel and skips the ones whose inputs did not change, which keeps the loop fast without splitting the repository.

Study first

  • pnpm workspaces and Turborepo give you one lockfile, one type check and one CI. Turborepo runs a task graph with caching, so a task whose inputs did not change does not run again. Agents need conventions they can follow: where things go, how packages import each other, and an instruction file per package.
  • The curriculum draws a monorepo with apps and packages. Before you draw it, ask whether a second app exists, or a second consumer of a package. One deployable was the decision of Day 26, and a package with one reader is a layer, not a boundary.

Build

  1. Write turbo.json with root tasks for generate:check, lint, typecheck, test and build, and with the site's lint, typecheck and build. Add the build's outputs, and the variables that the test and build hashes include.
  2. Make pnpm check run turbo run ..., and in CI cache .turbo by commit and restore it from the previous run.
  3. Measure the check cold and warm.

Done when

  • A warm pnpm check with nothing changed finishes in well under a second. In the reference run, it took 0.2 s, with every task restored. Cold, it took 17 s with the 8 tasks in parallel, against 32 s for the same work as a sequential chain.
The task graph
pnpm check through Turborepo. The first run does the work, and the second finds every task's inputs unchanged.
Technical notes
  • A root task is named //#task in turbo.json, and turbo run lint runs the root's lint and the site's when both are declared.
  • The root build's outputs name .next without its cache and its development state. The root test and build tasks hash CUSTOMER and DATABASE_URL into their keys, so a change of tenant is a different run.
  • The cold 17 s in the reference run had the Next.js compiler cache warm, and the first build on a machine takes longer.
  • docs/config.md describes the layout to adopt when a second app arrives, with the trigger, so the move is made once and for a reason.

Write up config-first

A third tenant, a new rule or a new entity should not need you in the room. So you finish by writing down how each one is done, and what the monorepo move waits for.

Study first

  • The write-up is a guide for the next change, not a diary. A reader should be able to follow it alone, and it should say plainly what is not built. The Diátaxis page on how-to guides describes that kind of document, with steps toward one goal and the explanation linked, not inlined.

Build

  1. Write docs/config.md, covering the split, the levels, the grammar and the proposal flow. Add the generator with its counts, the task graph with its timings, and what is not built.
  2. Record the decision in brain/architecture/decisions.md, D18 in the reference run. Then update the backlog, the README and the conventions in AGENTS.md, and update the routing and data-layer documents where they described the table in code or the JSON rules.

Done when

  • From the document alone, a reader can bring up a third tenant, add a rule and add an entity.
  • The reader knows what the monorepo move waits for.

Resources