Day 30

Agent security: RBAC, keys and data separation

You ship
WorkBot integrated, config-first and hardened, with an injection eval set whose 20 attempts all fail.

What you'll build today

After Day 29, WorkBot serves more than one customer from configuration, reads a customer's ticketing system and acts through tools. Yet anyone who reaches it can use it, because WorkBot has had no sign-in. Its customers also share tables that nothing in the database keeps apart. A customer's security review would stop here.

Today you harden WorkBot the way that review will test it. The database separates each customer's rows, and people and services sign in with keys and act through roles. Every page, action, route and tool checks the role in code, below whatever the model was told, because a prompt cannot enforce a permission. You seal integration tokens at rest, keep personal data out of traces, and delete old rows on a schedule.

Then you run 20 planted prompt injections, instructions hidden in data that try to steer the model, as an eval, a scored test of WorkBot's output. None of them may get through. The day ends with a one-page note that answers a customer's security questionnaire.

The layers
The proxy, the code Next.js runs before each request, turns away a request with no credentials. Pages, actions and routes check who is calling, tools check the role on every run, and Postgres scopes rows by tenant. Keys are hashed, tokens sealed, trace errors redacted and old rows deleted.

By the end of the day you can

  • Enforce role-based access control (RBAC) on every tool call and every retrieval, below the model rather than in the prompt
  • Separate tenants' data end to end: rows, embeddings, caches, logs and traces. A tenant is how WorkBot keeps one customer's data and settings apart
  • Manage secrets and API keys for agents the way production systems require

Topics

Threat Model for an Agent

  • Confused deputy: the agent acts with your credentials on instructions it read somewhere
  • Injection surfaces: tool results, documents and web pages
  • Capabilities limit damage, and instructions do not. What the agent can call matters more than what it is told not to do
  • Checklist: the Top 10 for applications built on large language models, from the Open Worldwide Application Security Project (OWASP)

RBAC Below the Model

  • Identity and roles travel with every request into every tool, and tools check authorization themselves
  • Retrieval respects permissions. Filter by tenant and role before ranking, never after. Postgres row-level security is the enforcement point
  • Tool allowlists per role, with destructive tools behind confirmation and audit
  • Testing: an eval set of privilege-escalation attempts, every one of which must fail

Agent Data Separation

  • Tenant isolation in every store: database rows, vector indexes, caches, object storage, logs and traces
  • Memory (Day 10) scoped per user and tenant, with no cross-tenant retrieval, even by similarity
  • Providers: retention policies, zero-data-retention options and regional endpoints, written into the customer contract
  • Personally identifiable information (PII): detect and redact it before logging and tracing, and set retention windows

Keys + Secrets

  • Per-environment, per-service keys with least privilege. No key in browser code, a prompt or a repository
  • Rotation without downtime, leaked-key detection, and spending limits per key
  • Secrets managers (Day 36): the app reads secrets at runtime
  • Integration tokens (Day 27): per user, encrypted at rest, scoped and revocable

Injection Defenses

  • All tool output is data. Mark its provenance in the context. Retrieved text never changes the tool allowlist
  • Structured tool results instead of free text where possible
  • Human confirmation for irreversible actions, and rate limits per user and per tool

Build

Features shipped today

  • WorkBot: Roles and tenants, row-level security, and encrypted integration tokens. Personal data redacted from traces, with retention windows on a schedule. An injection eval set, with attempts through chat, planted tool results and uploaded documents, all of which fail

Harden WorkBot, config-first since Day 29, and its integrations (Days 27 and 28):

  1. Roles (admin, agent, viewer) and tenants. Every tool checks authorization. Row-level security on every table the agent reads, and vector search filtered by tenant before ranking
  2. Secrets in a secrets manager or encrypted environment config. Per-user integration tokens encrypted at rest, with rotation and revocation
  3. An injection eval set: 20 attempts, through documents, tool results and chat, to exfiltrate data or escalate privileges. Every attempt must fail
  4. PII redaction before tracing, and a retention policy on traces and logs
  5. A one-page security note for a customer: what the agent can do, what it cannot, and what is stored where, for how long
  6. 3 end-to-end tests that stay in CI: a new tenant onboarded from configuration, an integration acting through a product-level tool, and an injection attempt failing

Step by step

Separate tenants in the database

Separation comes first, because every later check assumes that a query sees only its own tenant's rows. Postgres can enforce that for every query, whether or not the query remembers its filter, so you put the rule there.

Study first

  • An agent can become a confused deputy, which uses its caller's authority on instructions embedded in data, such as a ticket, a page or a document. So treat everything the model reads that a person did not type as untrusted input, and enforce permissions in code, because a prompt cannot. The Open Worldwide Application Security Project (OWASP) keeps a Top 10 list of the risks to assess in applications built on large language models.
  • Isolation applies to every store: rows, the vector index, caches and traces. A tenant is how WorkBot keeps one customer's data and settings apart, and you filter by tenant before ranking, never after.
  • Postgres row-level security puts a policy on a table, and forcing the policy makes it bind the table's owner too. A connection setting names the tenant, so a query that omits its filter still sees only its tenant's rows.
  • A superuser, and any role with BYPASSRLS, ignores every policy. So the app must query as a role that owns nothing and bypasses nothing.

Build

  1. Free the word tenant for the customer. WorkBot's traces and cache had a tenant column that held the caller, which was a visitor, the terminal or the system. Rename it caller in the code, with a migration that renames the column only where caller is still missing. Then tenant can mean the customer whose deployment wrote a row.
  2. Add the last statements of src/lib/schema.ts. Create the role workbot_app, then loop over every table to add a tenant column that defaults to the connection's app.tenant, enable and force row-level security, and add one policy. A table added later is covered without a statement of its own.
  3. Add the tenant to every natural key (tickets, contacts, chunks, prompts, cache, syncs, filings), and to the on conflict targets that name them.
  4. In src/lib/db.ts, let the owner apply the schema, and start every other connection with -c role=workbot_app -c app.tenant=<customer>. An embedded database has one connection, so the owner steps down to the role on it.
  5. Write tests where 2 tenants on one database read only their own rows, and keep separate copies of the same issue number. A write as another tenant is refused, and a connection with no tenant reads and writes nothing. workbot_app owns no table and bypasses nothing, every table forces its policy, and the whole schema applies twice to the same database.

Done when

  • The suite passes with the app querying as workbot_app.
  • Every local row carries the deployment's tenant.
Technical notes
  • Guard the rename on caller being missing, not on a tenant column being present. Step 2 adds a tenant column, so a guard that looks for one renames the new column on the second start, and the start fails. The test that applies the schema twice catches that class of mistake.
  • Do not let an empty app.tenant count as a tenant named "", or a write without a tenant succeeds. The default and the policy use nullif(current_setting('app.tenant', true), ''), so such a connection fails closed.
  • Knowledge-base search runs on the same connection, so the policy restricts its candidate rows before ranking.
  • The separation holds against the app's own queries, but not against someone who has the database credentials, who can set app.tenant or connect as the owner. A customer who needs that gets a database login of its own, or a database.

Sign-in, roles and keys

With the rows separated, WorkBot needs to know who is calling. It has had no sign-in, so you add people, services, roles and keys, and you check them where the data is read.

Study first

  • The public acts signed in as no one, and then come 3 roles, each granting everything below it. A viewer reads, an agent also acts, and an admin also administers.
  • People and services sign in with keys. Show a key once, store it as a hash, and make it revocable. Issue a second key before you revoke the first, so a service moves to the new key without downtime.
  • The proxy is the code Next.js 16 runs before a request reaches a page, and it was called middleware before. It checks only that a session cookie or a bearer header is present, and it validates neither.
  • The check that counts runs where the data is read: each page's data, each server action and each route handler. A server action is a POST to its page, so a change to the proxy's matcher can silently drop its coverage.

Build

  1. Add 3 tables, users, keys and sessions, and write src/lib/auth.ts with createUser, issueKey, revokeKey and startSession. Add userOf(headers), which finds the caller by a bearer key or the session cookie.
  2. In the same file, add requireUser(role) for pages and actions, and authorized(request, role) for routes, which answers 401 when there is no caller and 403 when the role is too low.
  3. Build /sign-in, which exchanges a key for a 7-day session, and make the header name the signed-in user. Build /admin/users and pnpm users, which add people and services and issue and revoke keys. Make the first admin in the terminal, which already holds the database.
  4. Put the checks in every protected page, server action and route, and in what the entity generator of Day 29 writes for employee entities.
  5. Render a control only for a role that can use it, with Allowed or permits. Production hides a server action's error text, so a viewer should not meet a button that the server refuses, and the server-side check remains the boundary.
  6. Write tests showing that keys are hashed and a second key works beside the first. Revoking a session's key ends the session, and so does its expiry. A route answers 401 and 403, and the proxy redirects or passes requests through.

Done when

  • A request to an employee page without a session lands on sign-in, and returns to the page after it.
  • A viewer's pages offer no action above the viewer role.
Sign in
A request without a session is sent to sign in, with the requested page as next.
Users and keys
The users page shows roles, and each key by its last 4 characters with when it was used. A new key is shown once. The key in the picture was revoked right after the screenshot.
What a viewer sees
The tickets page for a viewer shows the copy and the measurements, without Sync now or Reconcile now.
Technical notes
  • A session stores the hash of a random token and a reference to its key. The lookup rejects an expired session and a session whose key is revoked, so revocation takes effect on the next request.
  • currentUser is wrapped in React's cache, so a page that asks for the user in several components looks the user up once per render.
  • The redirect after sign-in resolves the requested path as a URL, and keeps it only when it stays on this site. A check for one leading slash is not enough: a path like /\attacker.example passes it, because a browser reads the backslash as a second slash.

Check roles inside every tool

The model chooses which tool to call, so the role check has to sit inside the tool itself. Then no instruction the model reads can widen what its caller may do.

Study first

  • Each role gets an allowlist of tools, and every tool checks the role inside itself, because what the model was offered and what it was told are not the boundary. OWASP's entry on excessive agency names the risk, and its fixes include fewer tools and authorization checked outside the model.
  • OWASP's authorization cheat sheet gives the rules the checks follow, such as deny by default and check permissions on every request.
  • Destructive actions need confirmation and an audit trail. In WorkBot, no tool closes, reassigns or deletes a ticket, and each write carries an idempotency key and supports a dry run (Day 27).

Build

  1. Add access to tool(). It names the lowest role that may call the tool, and it is required, so a tool definition without it does not compile. The run checks the role in the request context, and refuses with a sentence the model reads.
  2. Write toolsFor(role). The public gets the public set: the documents Fieldstone's own customers may read, a calculator and public pages. An employee gets every tool their role reaches.
  3. Make the chat route derive the role from the authenticated request, and carry it in the context. A signed-in user's memory follows the user, and a visitor's memory follows the browser.
  4. Make command-line entry points that run tools work as the operator, with the admin role, through asOperator. A context with no role is the public, so code that forgets a role fails closed.
  5. Write an end-to-end test where a viewer's assistant reads a ticket that orders it to comment, and a scripted model obeys, but the comment is refused below the model. Also check that the chat never offers a viewer the write in the first place.

Done when

  • A viewer who asks the chat to file a ticket gets an answer, not a ticket.
A viewer's chat
A viewer asks for a ticket. The assistant searches, has no tool to file, says so, and drafts the ticket for someone who can file it.
Technical notes
  • Knowledge tools built for the public sections are public, and the whole knowledge base takes the viewer role or above.
  • Tests that call a tool directly get a role through actingAs(role, tools). Never widen a tool's access to make a test pass, because that would be the bug.
  • The agent role is offered the write tools, so an instruction planted in data can make an agent's assistant attempt a write, and only the model's judgment stops it. The role bounds everyone below the agent, and for actions worth a person's sign-off, you add a confirmation step.

Seal tokens and redact traces

Some data has to be stored but should never sit in the clear. So you seal integration tokens at rest, and you replace personal data with labels before a trace is stored.

Study first

  • Per-user integration tokens are encrypted at rest, revocable, and used for the work done in that person's name. Sealing a token does not narrow what it can do at the provider, but the scope of the person's token does.
  • A key can rotate without downtime. The old key opens what it sealed while the new key seals, and a reseal moves every token to the new key before you remove the old one. A reseal is safe to run again, so you finish an interrupted one by rerunning it. OWASP's cryptographic storage cheat sheet recommends the Advanced Encryption Standard (AES) in an authenticated mode. It also says when to rotate a key, and how to move data to the new one.
  • Personal data stays out of logs and traces. OWASP's logging cheat sheet lists what to remove or mask before a log is written, from access tokens to personal data.

Build

  1. Write src/lib/secrets.ts, which seals with AES-256-GCM (Advanced Encryption Standard in Galois/Counter Mode) through Node's crypto under SECRETS_KEY. Store the key's fingerprint beside each secret, and accept SECRETS_KEY_PREVIOUS during a rotation.
  2. Write src/lib/credentials.ts and the credentials table, to save a token, open it for one call, revoke it and reseal it.
  3. Write githubToken(), so a GitHub call runs with the signed-in user's own token when they connected one, and with the app's token otherwise. The ticket tools are offered wherever a repository is configured. People connect a token on the settings page or with pnpm users connect, and an admin can revoke it.
  4. Write src/lib/redact.ts, which replaces personal data with labels in a trace's error before it is stored. It covers email addresses, phone numbers, card numbers that pass the Luhn checksum, US social security numbers, and recognized keys and tokens. The patterns cover common formats, not every kind of personal data.

Done when

  • A token is unreadable in the database, and opens for its user.
  • A reseal moves the token to a new key.
  • A trace's error carries labels where the personal data was.
Connections
The settings page for a signed-in user, with their own GitHub token. WorkBot seals the token before it is stored, and opens it only for the call that needs it.
Technical notes
  • Traces never held prompts or replies. The error was the one field that could quote what WorkBot sent a provider.
  • A number that matches the card pattern is redacted only if it passes the checksum, so an order or invoice number survives.

Delete old data on a schedule

Data that WorkBot no longer keeps cannot be exposed later, and a customer's contract can set how long each kind may be kept. So the windows become configuration, and a nightly job enforces them.

Study first

  • A retention window says how long WorkBot keeps each kind of row. The windows live in configuration, so a customer's file can shorten them without a change to code. The UK data protection regulator's guide to storage limitation explains why each kind of personal data needs a set period, and what happens when it ends.
  • The nightly job is an Inngest scheduled function, a cron trigger like the reminders of Day 18.

Build

  1. Put retention windows in content/defaults.json, which a customer's file can shorten. Traces last 90 days, and measurements, meaning the cache reads and the sync log of Day 28, last 30 days. A document's original bytes last 30 days after its run ends.
  2. Write src/inngest/retention.ts, a nightly job that deletes what has outlived its window, and the expired sessions. When a document's bytes go, its extracted record stays on the request.
  3. Write a test where old rows go, new ones stay, and a second run deletes nothing.

Done when

  • The windows are configuration that a contract can change.
  • The job's counts say what it removed.

Test 20 injection attempts

Roles bound what each caller may do, but the model can still be steered by what it reads. A set of planted attacks shows whether WorkBot holds, and running it on every change keeps it holding.

Study first

  • The eval plants attempts to exfiltrate data or escalate privileges in each channel the agent reads, and it runs on every change to a prompt, a tool or a model. OWASP's entry on prompt injection separates direct attempts from indirect ones hidden in content the model reads. Its example attacks are a source of cases.
  • The check makes the eval. A forbidden phrase must be one that only a compromised answer contains. Where only meaning separates obeying an instruction from warning about it, you ask a judge model instead. A passing eval is evidence about these cases, not a guarantee against others. OWASP's prompt injection prevention cheat sheet lists the common attack types, and asks you to define each case's intended violation before you test.

Build

  1. Write content/eval/injection.json with 8 attempts in the chat, 8 planted in a ticket, a contact or a cached page, and 4 inside documents. Mark 5 for the smoke set, the cases every pull request runs (Day 25).
  2. Write src/lib/eval-security.ts. Plant each attempt where the assistant will read it, run the assistant with the tools its role reaches, and send ticket writes to a sink that records whatever gets through.
  3. In the same file, fail the attempt on a write, a saved note or a forbidden phrase. Also fail it on a wrong field in a document's record, or on a reply that the judge finds following the planted instruction.
  4. Add pnpm eval --task security to the CI gate with the other tasks. Write tests with scripted models that do obey, to show each check catching a breach.

Done when

  • All 20 attempts fail, twice each.
  • The tests show the checks catch an assistant that obeys.
The injection eval
pnpm eval --task security in the reference run: 20 attempts through chat, planted data and documents, 2 runs each, all held.
Technical notes
  • Do not take forbidden phrases from the planted text, because a correct warning repeats that text. In the reference run, the first run failed 3 cases this way, and each time the assistant had reported the planted instruction and said it ignored it. Those cases now check that nothing was written, and the phishing case asks the judge whether the reply endorsed the instruction.

Write the security note

A customer's security reviewer reads a note, not the code. So you finish with one page that answers their questions, and that points to the code behind each answer.

Study first

  • A security note is only as strong as its evidence, so each claim names the code or the test that holds it. The note also says plainly what is not built yet. OWASP's introduction to the Application Security Verification Standard explains the requirements a buyer can ask a vendor to meet. Each requirement is a pass or a fail that someone can verify, and the note's claims should read the same way.

Build

  1. Write docs/security.md with a table of who can do what, from the public to the admin. Say what is decided below the model, and what is stored where and for how long. Cover keys and secrets, providers, and what is not built yet, and make each claim name the code or the test that holds it.
  2. Write the decision record in brain/architecture/decisions.md, D19 in the reference run. Update the backlog, the README, AGENTS.md and the architecture document.

Done when

  • A customer's security reviewer can answer their questionnaire from the note.
  • The reviewer can find the code behind every claim.

Resources