Unravel โบ Java concepts โบ APIs and web
Rate limiting and throttling
A rate limiter caps how many requests a client or operation may make per time window.
At a glance
A nightclub door.
- 1The token bucket algorithm: a bucket holds up to N tokens and is refilled at a steady rateEach request takes a token; with none left, the request is rejected or delayed. Bursts up to N are allowed, the long-run rate is the refill rate.
- 2In code, bucket.tryConsume(1) returns true if a token was availableLibraries (Bucket4j, Resilience4j RateLimiter, Guava) implement this with the clock handled for you.
- 3A rejected request should answer 429 Too Many Requests with a Retry-After header, so well-behaved clients know when to come back
- 4The key is chosen per client: API key, user id or IP addressOne bucket per key is kept in memory or in Redis.
- In-memory limits apply per instance
- Limit on the way out too
- Implementations store only the token count and the last refill timestamp
Bandwidth limit = Bandwidth.classic(100, Refill.greedy(100, Duration.ofMinutes(1)));Bucket bucket = Bucket.builder().addLimit(limit).build();if (!bucket.tryConsume(1)) { return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS) .header("Retry-After", "30") .build();}- line 1 uses Bandwidth, the core of Rate limiting and throttling
- line 4 uses tryConsume, the core of Rate limiting and throttling
Read the full explanation (about 2 min)
A rate limiter caps how many requests a client or operation may make per time window. It protects your service from overload and abuse, and protects other services from your own bursts.
A mental model
A nightclub door. The bouncer lets in a few people per minute; when the room is full, the rest wait outside or are turned away politely.
How it works
The token bucket algorithm: a bucket holds up to N tokens and is refilled at a steady rate. Each request takes a token; with none left, the request is rejected or delayed. Bursts up to N are allowed, the long-run rate is the refill rate.
In code, bucket.tryConsume(1) returns true if a token was available. Libraries (Bucket4j, Resilience4j RateLimiter, Guava) implement this with the clock handled for you.
A rejected request should answer 429 Too Many Requests with a Retry-After header, so well-behaved clients know when to come back.
The key is chosen per client: API key, user id or IP address. One bucket per key is kept in memory or in Redis.
Going deeper
In-memory limits apply per instance. With five replicas a "10 per second" limit becomes 50. A shared store (Redis) or the gateway is needed for a global limit.
Limit on the way out too. Calling a partner API faster than its quota gets your key blocked; an outbound limiter smooths that.
IP-based limits behind proxies see the proxy address unless you read the forwarded header from a trusted proxy only, and attackers rotate IPs, so combine with authentication-level limits.
Under the hood
Implementations store only the token count and the last refill timestamp. On each call they compute the tokens earned since then (elapsed time times rate), cap at capacity and subtract. Distributed versions do this atomically in a Redis Lua script or compare-and-swap, so no two instances double-spend a token.
Common mistakes
- Per-instance limits in a scaled-out service.
- Rejecting without a Retry-After hint.
- Limits keyed only by IP.
- Unbounded maps of buckets for every key (a memory leak).
How Unravel recognises it
When you paste code, Unravel looks for these signs of rate limiting and throttling. Spotting them yourself is a good habit when you read unfamiliar code:
- Annotations:
@RateLimiter - Types:
Bucket4jBandwidthRefillRateLimiterRateLimiterConfigRateLimiterRegistryRRateLimiterConsumptionProbeBucketConfiguration - Method calls:
tryConsume()tryAcquire() - Configuration keys:
Retry-AfterX-RateLimit-Remaining - Constants:
HttpStatus.TOO_MANY_REQUESTS
Part of a learning path
- Cloud-native Java ยท previous: API gateways (Spring Cloud Gateway)
Related concepts
Frequently asked questions
What is Rate limiting and throttling?
A rate limiter caps how many requests a client or operation may make per time window. It protects your service from overload and abuse, and protects other services from your own bursts.
What are the most common mistakes with Rate limiting and throttling?
Per-instance limits in a scaled-out service. Rejecting without a Retry-After hint. Limits keyed only by IP. Unbounded maps of buckets for every key (a memory leak).
How can I see Rate limiting and throttling in real code?
Paste your own Java into Unravel and it will point out where rate limiting and throttling appears, explain how it works at the detail level you choose, and draw the flow of the code around it.