Adaptive async rate limiting for Python

Closed-loop feedback control for API concurrency. When APIs push back, gentlify slows down. When pressure eases, it speeds up.

$ pip install gentlify

Adaptive Concurrency

Dynamic concurrency limits that decelerate on failures and cautiously recover after a cooling period of sustained success.

Dispatch Interval + Jitter

Enforces minimum time gaps between requests with stochastic jitter to prevent thundering-herd bursts.

Token-Aware Budgeting

Track per-window resource consumption — LLM tokens, API credits, bytes — independently of request-count limits.

Circuit Breaker

Hard stop after sustained failures with automatic half-open probing for safe recovery.

Zero Dependencies

Pure Python standard library. No runtime dependencies. Ships with py.typed and passes mypy --strict.

Progress & Observability

Real-time snapshots with ETA, structured event callbacks, and standard logging integration.

Quick Start

from gentlify import Throttle

throttle = Throttle(max_concurrency=5)

async def process(items):
    async with asyncio.TaskGroup() as tg:
        for item in items:
            tg.create_task(call_one(item))

async def call_one(item):
    async with throttle.acquire() as slot:
        result = await call_api(item)
        slot.record_tokens(result.usage.total_tokens)
    # Success/failure recorded automatically