Skip to main content
Independent of token quotas, several request-rate ceilings apply. From the edge inward:

​
1. Edge IP limit

The edge in front of inference.square1.dev rate-limits by client IP (on the order of 30 requests/minute with a small burst allowance). Exceeding it returns a plain edge error before the request reaches the API — you will not get the JSON error envelope. This exists to absorb runaway clients; normal chat usage doesn’t approach it.

​
2. Per-model RPM (yours)

Each model has a per-account requests-per-minute limit — read it from rate_limit_rpm in GET /v1/models and from the x-ratelimit-*-requests response headers.
  • Only served requests count toward the window; requests rejected by quota or validation don’t burn RPM.
  • Exceeding it → 429 rate_limited.

​
3. Account burst cooldown

A safety net across all models: if one account fires too many attempts (any status) in a short window, every request from that account is rejected for a cooldown period.
  • Trip → 429 abuse_cooldown with resets_at in the envelope.
  • The most common cause is an aggressive auto-retry loop hammering a failing request. Back off on errors instead of retrying in a tight loop — retrying into a cooldown only extends the outage on your side.

​
4. Model concurrency

Each model has a cap on concurrent in-flight requests across all users. When it saturates you get HTTP 529 model_overloaded — a global “this model is busy” signal, not something you caused. Retry after a few seconds.

​
Retry guidance