Many agent instances hit the same rate-limited API at once. Why does adding jitter to the backoff matter more than the exponent?

A tool call that returns a 404 should be retried with the same backoff policy used for a 429.

delay = min(base * 2^attempt, cap) * (0.5 + random()/2). The jitter matters more than the exponent when many agent instances retry at once, because synchronised retries reproduce exactly the overload that caused the failure.

University of Michigan - Ann Arbor

A topic of the Building AI Agents course. Planned for 2 hours at Hard difficulty.

Failure Handling in the Tool Loop

In a long trajectory, tool calls fail routinely: rate limits, stale ids, malformed arguments, empty results. An agent's competence is largely its competence at recovery, so failure paths deserve as much design attention as the happy path.

Failure Is the Normal Case

An idempotent tool can be called twice with the same arguments and leave the world in the same state. Since retries are unavoidable and the agent cannot always tell whether a timed-out call landed, write-side tools should be idempotent by construction, usually via a caller-supplied key.

Learn Before

Related

Learn After