When a model provider slowed down for 60 seconds in a simulation of 12,000 requests, a client with plain retries failed 18.5 percent of them and sent 1.62 provider calls for every user request. Adding
Fri Oct 23 2026 · 11 min read · 0 views
On 100,000 vectors, a pgvector HNSW index with a WHERE tenant = ... filter returned 3.6 percent of the true nearest neighbors, and still reported success. The same query on Qdrant found 99.95 percent.
Fri Oct 23 2026 · 10 min read · 0 views
Retrying a whole ten-step AI agent job after any failure wasted 4,143 model calls on 2,000 jobs, lost 75 jobs for good, and ran the email step twice for 273 jobs. Saving a checkpoint after each step
Fri Oct 23 2026 · 10 min read · 0 views
A plain usage counter overbilled $3,266.75 of AI usage by $114.86, a 3.5 percent error, because the queue delivered 6,984 events twice. All 500 tenants were billed wrongly. Recording each event once
Fri Oct 23 2026 · 9 min read · 0 views
In a shared FIFO queue, one tenant's burst of 2,000 AI jobs made every other tenant wait 9.6 seconds at the 95th percentile. A fair queue with no hard cap cut that to 27 milliseconds, and the noisy
Fri Oct 23 2026 · 10 min read · 0 views
One request to an AI support chat costs $0.0168 on Claude Sonnet 5.5 when every call resends the full prompt. Two changes cut that to $0.0035: prompt caching and sending easy questions to a cheaper
Fri Oct 23 2026 · 10 min read · 0 views