In a shared FIFO queue, one tenant's burst of 2,000 AI jobs made other tenants wait 9.6 seconds at the 95th percentile (our lab, 2026).
A fair queue with no concurrency cap cut that wait to 27 milliseconds while the noisy tenant still finished in 12.2 seconds, but a hard cap of 4 slowed it to 50.4 seconds and left 12 of 20 workers
Provider rate limits are shared across all tenants, so per-tenant budgets, a fair queue and queue limits that return 429 with Retry-After are needed in front of the model API.