Repository navigation
Job Retries with Backoff #2
Description
Goal: Automatically handle transient failures.
Automatically retry jobs that return an error. The system should allow users to define a retry policy, including the maximum number of attempts and an exponential backoff strategy to avoid "thundering herd" issues.
- wrapping the user's
Jobin a retry-aware function. - should support strategies: Constant, Linear, or Exponential backoff.
Question: Does it make sense to implement this logic inside the package itself, or should retries be handled dynamically at the service or API layer?
Activity
navyn13 commentedon Jan 6, 2026
Hi @iyashjayesh,
To better handle transient failures and avoid thundering-herd issues, it would be really useful if the retry policy also supported:
baseDelay – initial wait time before the first retry
maxDelay – upper bound on backoff delay (especially important for exponential backoff)
Why this matters:
Base delay lets users control how aggressively retries begin
Max delay prevents exponential backoff from growing too large and causing long stalls in job execution
This would make retry behaviour safer and more predictable in production while keeping the API flexible.
iyashjayesh commentedon Jan 7, 2026
@navyn13 Agreed! Those parameters are essential for production safety. I'll add both:
BaseDelay&MaxDelayThis will prevents runaway delays while keeping retries predictable. Thanks for the suggestion!
navyn13 commentedon Jan 7, 2026
@iyashjayesh Sounds good 👍 I can take this up and implement it.
iyashjayesh commentedon Jan 19, 2026
@navyn13 any luck on this? do you need any help?
pkhsaxena commentedon Oct 12, 2026
Hi @iyashjayesh, I've opened #7 to add retries with backoff, covering
BaseDelayandMaxDelayas discussed above.It also improves observability: task IDs and kinds, structured logging via
slog, anOnTaskDonehook, and vendor-agnostic metrics with Prometheus and OpenTelemetry adapters.