In a simulation of a 60-second model provider slowdown, naive retries failed 18.5 percent of 12,000 requests, sent 1.62 provider calls per request and took 21 seconds to recover (our simulation,
A circuit breaker cut provider calls to 0.83 per request and recovery to 12 seconds, and adding a fallback answer turned all 2,452 failed requests into degraded answers with zero errors shown to
Shorter timeouts with backoff did worse than naive retries because abandoned calls keep their provider slot and still cost money, so cancel upstream and set timeouts from measured latency.