Week 7 · Day 34 of 40 · Build AI

Cloud services: AWS, GCP, Azure and the neo-clouds

You ship: The monorepo deployed on a neo-cloud and on a major cloud, with cost estimates and a recommendation. Builds on: Day 29, Day 33.

By the end of the day you can

  • Navigate the 3 major clouds by the handful of services an AI product needs
  • Choose between a major cloud and a neo-cloud platform for a given customer, with a written rationale
  • Stand up WorkBot on a managed platform and on a major cloud in one day

Topics

The Services That Matter

  • Categories: compute (functions, containers, VMs), storage (object, block), data (managed Postgres, caches, queues), networking (load balancers, DNS, private networks), identity (IAM roles and policies), secrets and keys
  • AWS: Lambda, ECS on Fargate, EC2, S3, RDS, ElastiCache, SQS, IAM, KMS, VPC
  • GCP: Cloud Run, GKE, Compute Engine, Cloud Storage, Cloud SQL, Pub/Sub, IAM, Cloud KMS, VPC
  • Azure: Functions, Container Apps, VMs, Blob Storage, Azure Database for PostgreSQL, Service Bus, Entra ID, Key Vault, VNet
  • Managed model endpoints on each cloud: the data-residency and procurement reasons a customer may require them

Neo-Clouds

  • Supabase: Postgres with auth, storage, realtime, edge functions and pgvector; the fastest path to a complete backend
  • Neon: serverless Postgres with branching; a database per preview deployment
  • Render and similar: containers, workers, cron and managed Postgres with little configuration
  • Vercel for the web tier; where its limits push work elsewhere (Day 33)
  • Tradeoff: speed and simplicity vs control, compliance scope and egress cost

Choosing for a Customer

  • Existing estate: enterprise agreement, landing zone, security review process; deploying into the customer's own account is often the requirement
  • Data residency and regions; private connectivity to customer systems
  • Cost models: per request, per second, per instance; estimate a monthly bill before building
  • Portability: containers + Postgres + object storage keep most options open

Accounts, Billing + Guardrails

  • Separate accounts or projects per environment; budgets and alerts from day one
  • IAM least privilege for humans and for the app; no root credentials in use
  • Tagging and cost allocation per tenant and per feature

Build

Features shipped today

  • WorkBot: Deployed twice: on a neo-cloud (Supabase and Vercel, with a Neon branch per PR) and on a major cloud (ECS on Fargate or Cloud Run, managed Postgres, a queue, secrets)

Deploy the Week 6 monorepo twice:

  1. Neo-cloud: Supabase (Postgres, auth, storage) plus Render or Vercel for web and workers; a Neon branch per PR for preview environments
  2. Major cloud: the same system on AWS with ECS on Fargate for API and workers, RDS Postgres, S3, SQS and Secrets Manager (or GCP: Cloud Run, Cloud SQL, Cloud Storage, Pub/Sub, Secret Manager)
  3. Both deployments pass the Day 25 eval smoke suite and the Day 33 load test
  4. A cost estimate for each at 1,000 and at 100,000 monthly active users, with the 3 assumptions that drive it
  5. A one-page recommendation for a named customer type (regulated enterprise, mid-market SaaS, public sector): the decision and the 2 reasons

Walkthrough

Put WorkBot on a neo-cloud and on a major cloud, priced and argued for a real customer. The system becomes two container images and a database, runs on a stack shaped like a major cloud's, passes the eval smoke suite and the load test there, and is priced at 1,000 and 100,000 monthly users on three platforms. The deployments create billed resources in someone's account, so they are written as steps for the account owner to run.

The stack
The stack as a major cloud runs it: the web tier, two realtime instances, Inngest self-hosted and a browser service on one private network, with Postgres under them and the model's API outside.

Part 1

Map the services an AI product needs

Study first

  • The categories: compute (functions, containers, virtual machines), storage (object, block), data (managed Postgres, caches, queues), networking (load balancers, DNS, private networks), identity (roles and policies), and secrets.
  • Each major cloud's names for them: on AWS, Lambda, ECS on Fargate, S3, RDS, SQS, IAM and Secrets Manager; on Google Cloud, Cloud Run, Cloud Storage, Cloud SQL, Pub/Sub and Secret Manager; on Azure, Container Apps, Blob Storage, Azure Database for PostgreSQL, Service Bus, managed identities and Key Vault.
  • Managed model endpoints (Bedrock, Vertex AI, Microsoft Foundry): a regulated customer may accept a model only under its cloud's terms, in a named region. For each, check where inference runs, who processes the data, and which agreement applies.

Build

  1. A table of WorkBot's parts and where each runs on a neo-cloud, AWS, Google Cloud and Azure, in docs/hosting.md.
  2. Write down what is left out, and why. A cloud queue would duplicate Inngest, which already retries and orders the work. Object storage can wait: documents of about 100 KB kept 30 days come to about 20 GB at 100,000 users, which Postgres holds without strain.

Done when

  • Every part has a home on every platform, and every product left out has a reason.

Part 2

Package the system as images

Study first

  • Multi-stage builds: install once, build once, copy only what runs into the final image.
  • Next.js's standalone output, a server with only the files its routes import.
  • What goes in an image and what never does: code and the build's commit, yes; secrets and per-environment settings, no.

Build

  1. A Dockerfile with two targets from one dependency install. web is the standalone server with the static files and the content folder it reads at run time. tools holds the source and every dependency, for the realtime server and the operator's commands.
  2. A build argument, COMMIT_SHA, that /api/health and the eval records report.

Done when

  • The web image starts with nothing but environment variables, and its health check names the commit it was built from.
Technical notes
  • The web image is 532 MB and the tools image 3.2 GB. The tools image carries the musl builds of every native package beside the glibc ones, because the lockfile does not record a package's C library, so pnpm cannot filter them. The realtime server runs from it, which costs a slower first start, not memory.
  • Health had read the commit only from Vercel's variable, and the eval command called git, which no image contains. Both now read COMMIT_SHA first.
  • Cloud Run runs x86-64 images only; a build on Apple silicon must pass --platform linux/amd64.

Part 3

Run it shaped like a cloud

Study first

  • What changes when the parts are separate machines: addresses instead of localhost, a private network, and a short list of what is public.
  • A dependency on the machine is a dependency on the platform. A laptop has Chrome and git; a container has neither.

Build

  1. compose.yaml: the web image, two realtime instances, Postgres with pgvector, Inngest self-hosted with its own keys, and Playwright's browser server, on one network. INNGEST_BASE_URL points the app at the engine.
  2. Run the system there, and fix what breaks. The investigation and the document evals launched the machine's Chrome, which no container or function has: both now connect to a Playwright server when BROWSER_URL is set, and the browser service exposes no port.

Done when

  • Health is ok on the stack's database, the engine has synced the app's 10 functions, and a document sent to the API goes through the workflow.
Compose
Six containers, the two images' sizes, and the web container's health naming its commit.
Inngest
The self-hosted engine with WorkBot's functions, synced from the web container. The dashboard is the dev server's, but the engine rejects an event without its key.
Technical notes
  • A Playwright server and its client must share a major and minor version, so the browser image is pinned to the playwright-core version in package.json.
  • The investigation's limits on where the agent may browse still hold on a remote browser, because they are routes on the browser context.

Part 4

Keep long connections alive in a cloud

Study first

  • Load balancers end idle connections; AWS's default is 60 s. Deploys restart every instance, and a platform stops one with SIGTERM before it kills it.
  • Cookies and hosts: a cookie without a Domain goes back only to the host that set it, and a WebSocket's handshake carries cookies like any request.

Build

  1. The realtime server pings every socket each 30 s and ends one that did not answer the last ping. It exits at once on SIGTERM, since nothing on it needs finishing.
  2. A socket that reopens itself, src/lib/socket.ts, on both the caller's page and the staff page. A reconnecting call keeps its id, so it stays one call on the team's page.
  3. COOKIE_DOMAIN: when the realtime server has a host of its own, the session cookie covers a parent domain that holds WorkBot's hosts alone.

Done when

  • A socket that stops answering is ended within a minute, a page reopens its socket 2 s after it closes, and an instance restarts in 1 s instead of about 10.
Technical notes
  • On localhost the cookie problem cannot appear, because cookies ignore ports. On two cloud hostnames, the realtime server would not have recognized staff.
  • Serving both under one host, with a load balancer routing a path to the realtime service, avoids widening the cookie. On AWS that is a rule on the load balancer already there; elsewhere it is a load balancer added in front of both services.
  • A random id made during render fails Next's prerender. The call's id is made inside the effect, and the reconnecting socket reuses its address.

Part 5

Pass the evals and the load test on the stack

Study first

  • What a smoke suite proves about a deployment: the image, the database, the model route and the services the code reaches, together.

Build

  1. Run the eval smoke suite and pnpm load from the tools image against the stack: docker compose run --rm tools node --import tsx cli/<command>.ts. Against a deployment on Google Cloud the same commands run as Cloud Run jobs, as docs/deploy.md shows.

Done when

  • Every eval gate passes, and the load test loses nothing and keeps the steady senders out of the noisy one's backlog.
Evals
Chat 0.95, extract 0.89, triage 0.88 and security 1.00 in the tools container, every gate passed; the extraction rendered its invoices through the browser service.
Load
500 documents through the self-hosted engine: the four steady senders' 200 done within 97 s, the noisy sender's 300 in 368 s with its 100 broken PDFs in the dead letters, none lost.
Jobs
The containerised app's jobs page after the run: 100 dead letters from the noisy sender, and both event consumers up to date.

Part 6

Price it at two scales

Study first

  • The three ways platforms charge: per request, per second of compute or memory, per instance-hour. Free tiers, plan minimums, and a database standby that doubles the price.
  • Fixed costs that do not scale down: load balancers, NAT gateways, instances kept running.

Build

  1. Three assumptions: a monthly user has 20 turns and sends 2 documents; on the small models a turn costs $0.0008 and a document $0.001, measured from the traces; a turn holds a server about 8 s, and traffic is concentrated in working hours.
  2. Line items for Vercel with Supabase, Cloud Run with Cloud SQL, and ECS with RDS, at 1,000 and 100,000 users, in docs/hosting.md.
Monthly total1,000 users100,000 usersModel's share at 100,000
Neo-cloud$180$2,25380%
Google Cloud$240$2,28479%
AWS$262$2,39775%

Done when

  • Each total traces back to the assumptions, and you can say which line moves first when one of them changes.
Technical notes
  • At 1,000 users the platform is about 90% of the bill, mostly minimums. Inngest's minute heartbeat that delivers domain events is 86,400 executions a month at any traffic, past the free 50,000, so the job engine's $99 plan starts with the first user.
  • At 100,000 users the platforms are within 7% of each other and the model dominates the bill. The large model is priced at 20 times the small one: every turn on it would take the model's line from $1,800 to about $32,000.
  • A project per environment and labels on what takes them split the platform bill by app, environment and tier. The traces already split the model's by tenant and task.

Part 7

Recommend a platform for a named customer

Study first

  • What decides it: the customer's existing cloud estate and security review, data residency, regulation, who will operate the system, and how hard the choice is to undo.

Build

  1. One page in docs/hosting.md for Fieldstone Supply: the decision, the two reasons, what it gives up, and when the answer changes.
  2. Fieldstone gets the neo-cloud. Nobody there runs a network, and the move to a major cloud is a runbook, since the images and Postgres go anywhere. It gives up a database standby.
  3. Harbor Dental, the other customer in content/customers, gets Google Cloud: patient data needs a business associate agreement from every vendor, and one agreement from Google covers all the services.

Done when

  • Fieldstone's owner could approve it from the page alone, and the page says what would change the answer.

Part 8

Write the deployments

Study first

  • Accounts and projects per environment, a budget with alerts before anything runs, one identity for the services with access to their secrets and nothing else, and labels for cost.
  • Database branches for previews (Neon): a copy-on-write branch per pull request, made from a parent without the customer's data.

Build

  1. docs/deploy.md: what any target must provide, then Vercel with Supabase, Inngest Cloud and Render, a Neon branch per pull request through Neon's Vercel integration, and Google Cloud with Cloud Run, Cloud SQL and Secret Manager, every gcloud flag checked against the installed CLI.
  2. Record the platform requirements found. WorkBot sets its tenant and role when a session starts, takes advisory locks and listens, none of which a transaction pooler carries, so it needs a direct connection; it reads DATABASE_URL_UNPOOLED first where a platform sets both. Supabase's direct connection needs its IPv4 add-on for Vercel to reach it.

Done when

  • Someone holding the accounts can run the steps from top to bottom, and every requirement found today is in them.
Technical notes
  • Cloud Run reaches Cloud SQL through a socket path, which postgres 3.4.9 cannot take from a URL; the runbook gives the database a private address on the project's network instead.
  • Neon's integration deletes a preview branch when its Git branch is merged or deleted. This repository keeps its day branches, so their databases are deleted by hand.
  • Vercel made WebSockets generally available on 20 October 2026, under its functions' limits. A call can outlast those, so the realtime server stays a container.

Resources