Circuit Breaker

Track per-provider:model health across four states (healthy, degraded, down, throttled) to prevent cascading failures and minimize wasted latency.

Four-State Model

Each provider-model combination maintains its own independent health state:

              3 consecutive 5xx          5 consecutive 5xx
  ┌──────────┐  ──────────▶  ┌──────────┐  ──────────▶  ┌──────────┐
  │ healthy  │               │ degraded │               │   down   │
  │          │  ◀──────────  │          │  ◀──────────  │          │
  └──────────┘   success     └──────────┘   success     └──────────┘
                                                              │
                                                     30s probe │ success
                                                              ▼
                                                        back to healthy


                          429 received
  ┌──────────┐  ──────────────────────▶  ┌───────────┐
  │ any state│                           │ throttled │
  │          │  ◀──────────────────────  │           │
  └──────────┘     60s cooldown expires  └───────────┘

State Definitions

State	Condition	Behavior
healthy	No recent failures, or a successful probe	Requests routed normally
degraded	3+ consecutive 5xx errors	Requests still routed, but flagged. The router prefers this provider less if alternatives exist.
down	5+ consecutive 5xx errors	Requests blocked. One probe request allowed every 30 seconds to test recovery.
throttled	Received HTTP 429	Requests blocked for 60 seconds. Automatic recovery after cooldown.

Next Steps

Automatic Failover — what happens when a provider goes down
Provider Routing — how the router selects a provider
Upstream Providers — the infrastructure behind each model