Series · 6 parts
System design for AI-powered SaaS at solo scale: LLM unit economics, multi-tenant isolation, usage billing, durable agent jobs, RAG storage and scaling drills. Measured numbers, not interview diagrams.
Curriculum progress0 of 6 parts completed (0%)
Estimated remaining time: 60 minutes
Advertisement
One request to an AI support chat costs $0.0168 on Claude Sonnet 5.5 when every call resends the full prompt. Two changes cut that to $0.0035: prompt caching and sending easy questions to a cheaper
Key finding
A five-turn support chat costs $0.0168 per request on Claude Sonnet 5.5 with no optimisation, and 79 percent of that bill is input tokens (our model, 2026).
In a shared FIFO queue, one tenant's burst of 2,000 AI jobs made every other tenant wait 9.6 seconds at the 95th percentile. A fair queue with no hard cap cut that to 27 milliseconds, and the noisy
A plain usage counter overbilled $3,266.75 of AI usage by $114.86, a 3.5 percent error, because the queue delivered 6,984 events twice. All 500 tenants were billed wrongly. Recording each event once
Retrying a whole ten-step AI agent job after any failure wasted 4,143 model calls on 2,000 jobs, lost 75 jobs for good, and ran the email step twice for 273 jobs. Saving a checkpoint after each step
On 100,000 vectors, a pgvector HNSW index with a WHERE tenant = ... filter returned 3.6 percent of the true nearest neighbors, and still reported success. The same query on Qdrant found 99.95 percent.
When a model provider slowed down for 60 seconds in a simulation of 12,000 requests, a client with plain retries failed 18.5 percent of them and sent 1.62 provider calls for every user request. Adding
Advertisement
Continuous dispatch
Get notified when the next measured breakdown goes live. No pitch decks, no fluff, just raw numbers and what they mean.
More series are on the way.