Error Handling
Part of Workflow rate limits and throughput
Handling API throttling with controlled retries
Handle API throttling with provider retry timing, shared pacing, bounded attempts and a clear failure route.
When an API throttles a workflow, pause affected traffic, then follow the provider’s retry timing and retry within a defined attempt and time budget. Slowing one worker alone will not resolve a shared limit while other workers keep sending at the same rate.
Read the response before choosing a delay
HTTP 429 means too many requests. The response may include Retry-After, but that header is optional, and its value is either a number of seconds or an HTTP date.
Check the provider’s documentation as well as the response: some APIs use other status codes for limits. GitHub’s REST API documentation, for example, describes primary rate limits that vary by authentication method and by endpoint.
Record the status, provider error code, relevant rate-limit headers, request ID and the affected credential or endpoint. Check which other workflows share the allowance. Apply any documented reset time or retry delay using its stated units and time basis. Where the provider gives no usable timing, use a bounded backoff policy with randomisation; never override an explicit wait instruction with a shorter fallback.
API Throttling Response Handling Strategies
- Documented throttle with usable timingPause traffic until provider permits retry, subject to workflow deadline. Use `Retry-After` header or documented reset time.
- Throttle without usable timingApply bounded backoff with randomisation; reduce new request rate. Never override explicit wait instructions.
- Validation or permission rejectionStop request and route for correction. Do not retry without fixing the cause.
- Timeout after write may have succeededCheck destination outcome or use idempotency mechanism before repeating the write.
Control the shared request stream
Place pacing ahead of the constrained API for all affected workers, or coordinate them through a dispatch queue. Reduce new requests while throttling persists. Set both a maximum attempt count and an overall deadline based on the action’s value.
Check whether the HTTP client or SDK already retries. Workflow-level retries added on top can multiply outbound attempts. AWS SDKs document retry modes, backoff and retry quotas, but their behaviour depends on the SDK configuration and whether newer retry behaviour has been enabled. Count attempts at the outbound boundary rather than relying only on workflow run counts.
| Outcome | Workflow response |
|---|---|
| Documented throttle with usable timing | Pause affected traffic until the provider permits another attempt, subject to the workflow deadline. |
| Throttle without usable timing | Use bounded backoff with randomisation and reduce the new-request rate. |
| Validation or permission rejection | Stop this request and route it for correction. |
| Timeout after a write may have reached the server | Check the destination outcome or use a supported idempotency mechanism before repeating the write. |
End retries deliberately
When the budget expires, retain the record and last error in a visible exception route with an owner. Log enough to identify the request without copying credentials or sensitive payloads. Monitor throttled responses, outbound attempts per completed action, waiting time and records past their deadline. A falling 429 count matters only if completed actions also recover.
Key Metrics to Monitor for Throttled APIs
- Throttled responses (429)
- Count over time
- Outbound attempts per completed action
- Track ratio to detect inefficiency
- Waiting time under throttling
- Measure average delay during peak load
- Records past deadline
- Monitor for unresolved cases
![How to choose the right idempotency key for an event: Keep the same idempotency key for retries of one action; use a new key for a new action.; Write: one [effect] for each [identity], e.g. one standard invoice per approved order.; Check stability, uniqueness, scope and lifetime; include tenant ID for tenant-scoped IDs. Choosing an idempotency key for an event](/covers/choosing-an-idempotency-key-for-an-event-640.webp?v=39704930)