Week 6 · Day 30 of 40 · Build AI

Agent security: RBAC, keys and data separation

You ship: WorkBot integrated, config-first and hardened, with an injection eval set it fails. Builds on: Day 10, Day 27, Day 29.

By the end of the day you can

  • Enforce role-based access on every tool call and every retrieval, below the model rather than in the prompt
  • Separate tenants' data end to end: rows, embeddings, caches, logs, traces
  • Manage secrets and API keys for agents the way production systems require

Topics

Threat Model for an Agent

  • Confused deputy: the agent acts with your credentials on instructions it read somewhere
  • Injection surfaces: tool results, documents, web pages
  • Capabilities limit damage, instructions do not: what the agent can call matters more than what it is told not to do
  • Checklist: OWASP Top 10 for LLM applications

RBAC Below the Model

  • Identity and roles travel with every request into every tool; tools check authorization themselves
  • Retrieval respects permissions: filter by tenant and role before ranking, never after; Postgres row-level security as the enforcement point
  • Tool allowlists per role; destructive tools behind confirmation and audit
  • Testing: an eval set of privilege-escalation attempts the agent must fail

Agent Data Separation

  • Tenant isolation in every store: database rows, vector indexes, caches, object storage, logs, traces
  • Memory (Day 10) scoped per user and tenant; no cross-tenant retrieval, even by similarity
  • Providers: retention policies, zero-data-retention options, regional endpoints; written into the customer contract
  • PII: detect and redact before logging and tracing; retention windows

Keys + Secrets

  • Per-environment, per-service keys with least privilege; no key in a client, a prompt or a repository
  • Rotation without downtime; leaked-key detection; spending limits per key
  • Secrets managers (Day 36): the app reads at runtime
  • Integration tokens (Day 27): per user, encrypted at rest, scoped, revocable

Injection Defenses

  • All tool output is data; mark provenance in the context; retrieved text never changes the tool allowlist
  • Structured tool results instead of free text where possible
  • Human confirmation for irreversible actions; rate limits per user and per tool

Build

Features shipped today

  • WorkBot: Roles and tenants, row-level security, encrypted integration tokens; personal and financial data as special categories: consent, retention, redaction, residency; injection attempts through tool results and uploaded documents

Harden the monorepo (Day 29) and its integrations (Days 27 and 28):

  1. Roles (admin, agent, viewer) and tenants; every tool checks authorization; row-level security on every table the agent reads; vector search filtered by tenant before ranking
  2. Secrets in a secrets manager or encrypted environment config; per-user integration tokens encrypted at rest, with rotation and revocation
  3. An injection eval set: 20 attempts (through documents, tool results and chat) to exfiltrate data or escalate privileges; the agent fails all of them
  4. PII redaction before tracing; a retention policy on traces and logs
  5. A one-page security note for a customer: what the agent can do, what it cannot, what is stored where, for how long
  6. 3 end-to-end tests that stay in CI: a new tenant onboarded from configuration, an integration acting through a product-level tool, and an injection attempt failing

Walkthrough

Harden WorkBot the way a customer's security review will test it. Each customer's rows are separated by the database; people and services sign in with keys and act through roles; every page, action, route and tool checks the role in code, below whatever the model was told; integration tokens are sealed at rest; traces keep no personal data and old rows are deleted on a schedule; and 20 planted injections run as an eval, none of which may get through. WorkBot had no sign-in before today. The day ends with a one-page note that answers a customer's security questionnaire.

The layers
The proxy turns away a request with no credentials; pages, actions and routes check who is calling; tools check the role on every run; Postgres scopes rows by tenant. Keys are hashed, tokens sealed, trace errors redacted, old rows deleted.

Part 1

Tenants, separated by the database

Study first

  • The threat model for an agent: a confused deputy uses its caller's authority on instructions embedded in data, a ticket, a page, a document. Treat everything the model reads that a person did not type as untrusted input, and enforce permissions in code: a prompt cannot. The OWASP Top 10 for LLM applications lists the risks to assess.
  • Isolation in every store: rows, the vector index, caches, traces. Filter by tenant before ranking, never after.
  • Postgres row-level security: a policy on a table, forced so it binds the table's owner too, and a connection setting that names the tenant. A query that omits its filter still sees only its tenant's rows. A superuser, and any role with BYPASSRLS, ignores every policy, so the app must query as a role that owns nothing and bypasses nothing.

Build

  1. Free the word. WorkBot's traces and cache had a tenant column that held the caller, a visitor, the terminal or the system; rename it caller, in the code and with a migration that renames the column only where it is still missing, so tenant can mean the customer whose deployment wrote a row.
  2. The last statements of src/lib/schema.ts: create workbot_app, then loop over every table to add a tenant column that defaults to the connection's app.tenant, enable and force row-level security, and add one policy. A table added later is covered without a statement of its own.
  3. Add the tenant to every natural key (tickets, contacts, chunks, prompts, cache, syncs, filings) and to the on conflict targets that name them.
  4. src/lib/db.ts: the owner applies the schema; every other connection starts with -c role=workbot_app -c app.tenant=<customer>. An embedded database has one connection, so the owner steps down to the role on it.
  5. Tests: two tenants on one database read only their own rows and keep separate copies of the same issue number; a write as another tenant is refused; a connection with no tenant reads and writes nothing; workbot_app owns no table and bypasses nothing; every table forces its policy; the whole schema applies twice to the same database.

Done when

  • The suite passes with the app querying as workbot_app, and every local row carries the deployment's tenant.
Technical notes
  • The rename's first guard looked for a tenant column, which step 2 adds, so the second start would have renamed the new column and failed; the schema-twice test catches that class of mistake.
  • An empty app.tenant first counted as a tenant named "", so a write without one succeeded; the default and the policy use nullif(current_setting('app.tenant', true), ''), and such a connection fails closed.
  • Knowledge-base search runs on the same connection, so the policy restricts its candidate rows before ranking.
  • The separation holds against the app's own queries, not against someone holding the database credentials, who can set app.tenant or connect as the owner. A customer who needs that gets a database login of its own, or a database.

Part 2

People, roles and keys

Study first

  • Who can act: the public, signed in as no one, then three roles, each granting everything below it. A viewer reads, an agent also acts, an admin also administers.
  • Keys for people and services: shown once, stored as a hash, revocable, with a second key issued before the first is revoked so a service moves without downtime.
  • In Next.js the proxy checks only that a session cookie or a bearer header is present; it does not validate either. The check that counts runs where the data is read: each page's data, each server action, each route handler. A server action is a POST to its page, so a matcher change can silently drop the proxy's coverage.

Build

  1. Three tables, users, keys and sessions, and src/lib/auth.ts: createUser, issueKey, revokeKey, startSession; userOf(headers) for a bearer key or the session cookie; requireUser(role) for pages and actions; authorized(request, role) for routes, answering 401 for no caller and 403 for too low a role.
  2. /sign-in exchanges a key for a 7-day session; the header names the signed-in user; /admin/users and pnpm users add people and services and issue and revoke keys. The first admin is made in the terminal, which already holds the database.
  3. The checks in every protected page, server action and route, and in what the entity generator of Day 29 writes for employee entities.
  4. Pages render a control only for a role that can use it, with Allowed or permits; production hides a server action's error text, so a viewer should not meet a button the server refuses. The server-side check remains the boundary.
  5. Tests: keys hashed, a second key beside the first, a session ended by its key's revocation or its expiry, 401 and 403 from a route, the proxy's redirect and its pass-through.

Done when

  • A request to an employee page without a session lands on sign-in and returns after it, and a viewer's pages offer no action above the viewer role.
Sign in
A request without a session is sent to sign in, with the requested page as next.
Users and keys
The users page: roles, each key by its last 4 characters with when it was used, and a new key shown once. The key in the picture was revoked right after it was taken.
What a viewer sees
The tickets page for a viewer: the copy and the measurements, without Sync now or Reconcile now.
Technical notes
  • A session stores the hash of a random token and a reference to its key; the lookup rejects an expired session and one whose key is revoked, so revocation takes effect on the next request.
  • currentUser is wrapped in React's cache, so a page that asks in several components looks the user up once per render.
  • The redirect after sign-in resolves the requested path as a URL and keeps it only on this site: a path like /\attacker.example passed a check for one leading slash, because a browser reads the backslash as a second.

Part 3

Tools, checked below the model

Study first

  • Tool allowlists per role, and a check inside every tool: what the model was offered and what it was told are not the boundary.
  • Confirmation and audit for destructive actions. Here no tool closes, reassigns or deletes a ticket, and each write carries an idempotency key and supports a dry run.

Build

  1. access in tool(): the lowest role that may call the tool, required, so a tool definition without one does not compile. The run checks the role in the request context and refuses with a sentence the model reads.
  2. toolsFor(role): the public gets the public set, the documents Fieldstone's own customers may read with a calculator and public pages; an employee gets every tool their role reaches. The chat route derives the role from the authenticated request and carries it in the context; a signed-in user's memory follows the user, a visitor's the browser.
  3. Command-line entry points that run tools work as the operator, with the admin role, through asOperator. A context with no role is the public, so code that forgets one fails closed.
  4. An end-to-end test: a viewer's assistant reads a ticket that orders it to comment, a scripted model obeys, and the comment is refused below the model; and the chat never offers a viewer the write in the first place.

Done when

  • A viewer asking the chat to file a ticket gets an answer, not a ticket.
A viewer's chat
A viewer asks for a ticket: the assistant searches, has no tool to file, says so, and drafts the ticket for someone who can.
Technical notes
  • Knowledge tools built for the public sections are public; the whole knowledge base takes the viewer role or above.
  • Tests that call a tool directly get a role through actingAs(role, tools); widening a tool's access to make a test pass would be the bug.
  • On the agent role the write tools are offered, so an instruction planted in data can make an agent's assistant attempt a write, and only the model's judgment stops it. The role bounds everyone below the agent; for actions worth a person's sign-off, add a confirmation step.

Part 4

Secrets at rest: sealed tokens, redacted traces

Study first

  • Per-user integration tokens: encrypted at rest, revocable, and used for the work done in that person's name. Sealing them here does not narrow what the token can do at the provider; the person's token scope does.
  • Rotation without downtime: the old key opens what it sealed while the new one seals, and a reseal moves every token to the new key before the old one is removed. A reseal is safe to run again, so an interrupted one is finished by rerunning it.
  • Personal data out of logs and traces.

Build

  1. src/lib/secrets.ts: AES-256-GCM under SECRETS_KEY, with the key's fingerprint stored beside each secret and SECRETS_KEY_PREVIOUS accepted during a rotation. src/lib/credentials.ts and the credentials table: save, open for one call, revoke, reseal.
  2. githubToken(): a GitHub call runs with the signed-in user's own token when they connected one, and with the app's otherwise; the ticket tools are offered wherever a repository is configured. People connect a token on the settings page or with pnpm users connect, and an admin can revoke it.
  3. src/lib/redact.ts: email addresses, phone numbers, card numbers that pass the Luhn checksum, US social security numbers and recognized keys and tokens are replaced by labels in a trace's error before it is stored. The patterns cover common formats, not every kind of personal data.

Done when

  • A token is unreadable in the database, opens for its user, moves to a new key with a reseal, and a trace's error carries labels where the data was.
Connections
The settings page for a signed-in user: their own GitHub token, sealed before it is stored and opened only for the call that needs it.
Technical notes
  • Traces never held prompts or replies; the error was the one field that could quote what a provider was sent.
  • A number matching the card pattern is redacted only if it passes the checksum, so an order or invoice number survives.

Part 5

Forget on a schedule

Build

  1. Retention windows in content/defaults.json, which a customer's file can shorten: traces 90 days; measurements, the cache reads and sync log of Day 28, 30 days; a document's original bytes 30 days after its run ends.
  2. src/inngest/retention.ts: a nightly job that deletes what has outlived its window and the expired sessions; a document keeps its extracted record on the request when its bytes go.
  3. A test: old rows go, new ones stay, and a second run deletes nothing.

Done when

  • The windows are configuration a contract can change, and the job's counts say what it removed.

Part 6

Twenty attempts that must fail

Study first

  • An eval of exfiltration and privilege-escalation attempts through each channel the agent reads, run on every change to a prompt, a tool or a model.
  • The check makes the eval. A forbidden phrase must be one only a compromised answer contains; where only meaning distinguishes obeying an instruction from warning about it, ask a judge. A passing eval is evidence about these cases, not a guarantee against others.

Build

  1. content/eval/injection.json: 8 attempts in the chat, 8 planted in a ticket, a contact or a cached page, 4 inside documents; 5 in the smoke set.
  2. src/lib/eval-security.ts: plant each attempt where the assistant will read it, run the assistant with the tools its role reaches, send ticket writes to a sink that records whatever gets through, and fail the attempt on a write, a saved note, a forbidden phrase, a wrong field in a document's record, or a reply the judge finds following the planted instruction.
  3. pnpm eval --task security, in the CI gate with the other tasks, and tests with scripted models that do obey, to show each check catching a breach.

Done when

  • All 20 attempts fail, twice each, and the tests show the checks catch an assistant that obeys.
The injection eval
pnpm eval --task security: 20 attempts through chat, planted data and documents, 2 runs each, all held.
Technical notes
  • The first run failed 3 cases, each one the assistant reporting the planted instruction and saying it ignored it: the forbidden phrases came from the planted text, which a correct warning repeats. Those cases now check that nothing was written, and the phishing case asks the judge whether the reply endorsed the instruction.

Part 7

The note for the customer

Build

  1. docs/security.md: who can do what, as a table from the public to the admin; what is decided below the model; what is stored where and for how long; keys and secrets; providers; what is not here yet. Each claim names the code or the test that holds it.
  2. Write decision D19 in brain/architecture/decisions.md, and update the backlog, the README, AGENTS.md and the architecture document.

Done when

  • A customer's security reviewer can answer their questionnaire from the note, and can find the code behind every claim.

Resources