A published research dossier, the analytics stack with a semantic layer, and a read-only analytics agent.
What you'll build today
WorkBot now has a site, a target list and a way to reach buyers. What it lacks is a way to know how it is used and what its market looks like, with numbers you can trust. An agent that answers questions about data must not invent numbers, so it has to show the query behind every figure.
Today you research one question about WorkBot's market with an agent, and publish the result with every claim's sources and a confidence label. Then you choose an analytics stack sized to the data, write the metrics once as SQL with tests, and build an analyst agent that answers in SQL it shows you. You learn how to keep research and numbers checkable, with sources for every claim, a definition for every metric, and known answers to test against.
By the end of the day, you have a published research dossier, metrics whose data tests pass, and an analyst that matches three known answers.
By the end of the day you can
Run first-hand research with coding agents: gather, verify, cite, publish
Build an analytics stack from Postgres to ClickHouse with Metabase or Hex on top, and size it to the data
Use agents for business intelligence (BI) without letting them invent numbers
Topics
First-Hand Research with Agents
Research as an agent workflow: a question, then a source list, then extraction per source with citations, then cross-checking, then a dossier with confidence labels
Claude Code and Codex as research tools: scripts to harvest, parse and tabulate. The repository is the lab notebook
Verification: every number traced to a source and a date, with disagreements between sources kept
Publishing: datasets with licenses, methodology sections, and stable URLs updated in place (Day 38)
BI Techniques
Metrics as definitions in code: one source of truth for revenue, active users and retention. These definitions are the semantic layer the agent queries
Modeling: events, entities, facts and dimensions. Decide what stays in Postgres and what moves to a column store
The 5 analyses every product needs before a dashboard: cohorts, funnels, retention curves, unit economics, segment comparison
Dashboards that answer a question, with the question in the title
The Stack
Postgres: operational truth and small analytics, with read replicas for queries
ClickHouse: event-scale analytics. It brings columnar storage, materialized views, and ingestion from Postgres and from event streams (Day 33)
Metabase: team dashboards and questions over both databases
Hex: notebooks that mix SQL, code and narrative, with agent-assisted analysis and review
Transformations as versioned SQL with data tests (dbt or equivalent)
Agents in the Loop
The agent writes SQL against the semantic layer, shows the query, runs it and cites the result. No number without a query
Scheduled analyses: weekly reports the agent writes from fresh queries, reviewed by a person before sending
Build
Features shipped today
WorkBot: The analytics stack over each customer's workflows, with cohort and retention analytics over anonymized user data. A research dossier on its market
Build the research and analytics layer for WorkBot:
Research: a market question relevant to WorkBot, with 50+ sources harvested by scripts. Extract claims and cross-check them. Publish a dossier page with citations, confidence labels and a dataset
Stack: ClickHouse ingesting product events from the Day 33 event stream and a nightly copy of the Postgres tables. Metabase connected to both, and Hex for notebooks. Size it to the data: at a few thousand rows, the metric files can run in the locked DuckDB from Day 31 and move to ClickHouse as written
Semantic layer: metric definitions in versioned SQL with data tests, and the 5 core analyses as dashboards
Agent BI: a read-only analytics agent with query display and row and cost limits. It produces the weekly report, which a person reviews
Prove it: 3 questions whose answers you know. Every answer shows its SQL and matches
Step by step
1
Research a market question
Research comes first, because it shows what WorkBot's market looks like before you measure WorkBot itself. An agent can harvest and cross-check the sources, as long as every claim keeps its sources and a label that says how well they support it.
Study first
Learn to harvest sources with scripts and agents, extract claims, and cross-check each claim against the others. Then label each claim's confidence by what supports it.
Read Anthropic's account of its multi-agent research system. Note how it taught its agents to prefer primary sources, and how a separate step ties each claim to its source.
Read the Intergovernmental Panel on Climate Change's guidance on confidence, which rates a finding by its evidence and by how far the sources agree. You should be able to say why two independent sources earn more confidence than one.
Build
1
Pick one question about WorkBot's market. In the reference run, it was how independent building-supply dealers take contractor orders and handle supplier invoices, with what software and at what cost.
2
Have an agent harvest the sources, extract claims and cross-check them, then give each claim a confidence label. High needs two independent sources or a survey with a method, medium needs one credible source, and low is a vendor's claim. In the reference run, the agent harvested 76 sources and extracted 47 claims.
3
Publish the dossier at /research on the public site, with the dataset at /research/dossier.json.
Done when
Every claim shows its sources and its label, and the dossier names its weakest finding as such.
The reference run's dossier on the public site gives the question, then a summary with each finding's confidence, then 47 claims with their sources.Technical notes
Label the parts that no good data supports. In the reference run, the weakest part was spend. No survey measures what dealers pay for software, and the only figures are third-party estimates, labeled low.
Some sites block automated reading. In the reference run several trade publications did, so some claims rest on search excerpts, and each such claim's note says so.
2
Choose the analytics stack
Before you write any metric, decide where the data lives and where the queries run. The stack should fit the data you have, because every extra system is one more thing to run.
Study first
Learn the parts of an analytics stack: warehouses such as ClickHouse, business intelligence (BI) servers such as Metabase, and notebooks such as Hex. Then learn the semantic layer, which defines each metric once in versioned SQL with data tests, as the dbt Semantic Layer does.
Build
1
Choose the stack for the size of the data. The curriculum's stack is ClickHouse with Metabase and Hex, and those come in when the volume calls for them. In the reference run, at two tenants (two customers' data) and a few thousand rows, the analytics ran in DuckDB in memory. That is the spreadsheets' sandbox from Day 31, generalized to load any data, with no file or network access.
2
Make a synthetic dataset in which a hash of person and day decides each row. Every run then has the same rows, so the known answers stay known, and people are ids, never names.
Done when
You can say what would make a warehouse worth running, and the metrics would move to it as written.
3
Metrics as SQL with tests
With the stack chosen, define each metric once. These definitions are the semantic layer, and data tests check the dataset and the definitions together, so a wrong number shows up as a failing test.
Study first
Read dbt's page on data tests for the convention your tests follow, without dbt itself. A test is a query that returns the failing rows, and it passes when it returns none.
Build
1
Write five files in content/analytics/metrics, for weekly active people, retention by weekly cohort, use by kind, spend per person and new people. Each file's first line is its title.
2
Write data tests in content/analytics/tests, each a query that must return no rows. One checks that nobody was active before joining, and another checks that no cohort retains more people than it had.
3
Add pnpm analytics report, which prints every metric's table.
Done when
The tests pass, and changing a definition changes the report and the analyst together.
4
An analyst that shows its SQL
The metrics answer the questions you thought of in advance. The analyst agent answers new questions in plain words, and it stays checkable because it can only run read-only queries and must show each one.
Study first
Learn what agent BI needs, where an agent answers questions about the data. It needs read-only access, query display, row and cost limits, and checks against known answers.
Read the Open Worldwide Application Security Project's entry on excessive agency, whose example is an agent that needs only read access to one table. Then read Anthropic's advice on writing tools for agents, which caps what a tool returns and tests each tool with evals.
Build
1
Write src/lib/analysis.ts, which gives the model one tool. The tool runs a single SELECT and returns 50 rows at most, with a time limit per query and step and dollar limits per question. Put the metric definitions in the agent's instructions, so that "retention" means what the team means.
2
Write three questions whose answers you know from hand-written SQL, and check that the analyst matches each one.
3
Add the analyst's four-sentence summary to the weekly report, and have a person read it before it goes anywhere.
Done when
All three known answers match, each with its SQL shown.
In the reference run, three of three matched: 300 people, 546 chats in September and $0.42 for Harbor Dental. Each answer shows the query that produced it.The data tests pass, and the report ends with the analyst's summary for a person to review.Technical notes
Make the dataset with a hash, not random(). In the reference run, DuckDB's random() gave the same draw to every cohort across the seed's cross join. A hash of person and day gives an independent, reproducible draw per row.
Run the check as the operator. The query tool requires a viewer's role, which WorkBot has checked outside the model since Day 30. In the reference run, the first run of the check failed safe because the terminal had not said who it was. The check now runs as the operator, like every other command.
