Your Rate Limiter Is Lying to You: Per-Instance Counters Silently Break at Scale

The Engineering Mistake

Product says: 100 requests per minute per user. You write this:

const counts = new Map();

function isRateLimited(userId) {
  const window = Math.floor(Date.now() / 60_000);
  const key = `${userId}:${window}`;
  const count = (counts.get(key) ?? 0) + 1;
  counts.set(key, count);
  return count > 100;
}

Works in dev. Ships on a Tuesday. You scale to four instances and users suddenly get 400 requests per minute. Each instance tracks its own counter. The load balancer splits traffic evenly. No single instance ever sees the limit.

The fixed-window variant makes it worse. A user can send 200 requests at the minute boundary: 100 in the last second of one window, 100 in the first second of the next. Neither window records a violation.

The user sent 100 requests. Both instances report count = 50. Both say you're fine.

The Fix: Shared Sliding Window in Redis

Move the counter out of process memory and into a shared store. Redis gives you atomic operations, sub-millisecond reads, and automatic key expiry.

The algorithm is a sliding window using a sorted set. Each request inserts a timestamped entry; you count entries from the last 60 seconds:

ZADD ratelimit:{userId} <now_ms> <request_uuid>
ZREMRANGEBYSCORE ratelimit:{userId} 0 <now_ms - 60000>
count = ZCARD ratelimit:{userId}
EXPIRE ratelimit:{userId} 60

ZADD records the current timestamp as the score. ZREMRANGEBYSCORE evicts anything outside the window. ZCARD returns the live count. EXPIRE cleans up keys for users who go quiet.

There's a race between ZADD and ZCARD: another instance could insert a request in between and you'd return a stale count. Wrap it in a Lua script and Redis runs the whole block atomically:

local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local uuid = ARGV[3]

redis.call('ZADD', key, now, uuid)
redis.call('ZREMRANGEBYSCORE', key, 0, now - window)
local count = redis.call('ZCARD', key)
redis.call('EXPIRE', key, math.ceil(window / 1000))

return count

Call this from your middleware, check if count > limit, return 429. Every instance hits the same Redis key. The counter is shared and the limit holds regardless of pod count.

Return X-RateLimit-Remaining in your response headers. The sorted set gives it to you for free: limit - count. Add Retry-After on 429s so clients can back off without polling your endpoint.

Tradeoffs

**Latency.** Redis adds roughly 1ms per request on a co-located instance, more if it's cross-region. For most APIs this disappears into noise. At sub-millisecond p99 requirements, instrument it before deciding.

**Redis on the critical path.** If Redis goes down, the limiter either blocks all traffic or passes it all through. Most teams choose fail-open: a few minutes of uncapped traffic during an outage beats a complete service blackout. Write the policy down explicitly. It's a deliberate choice, not an oversight.

**Memory scales with request volume.** A user hitting 100 req/min keeps 100 sorted-set entries alive. At millions of active users, that's real memory pressure. A token bucket using atomic INCR uses constant memory per user, at the cost of slightly less accurate burst protection.

**Edge rate limiting removes the Redis round trip entirely.** If you run behind Cloudflare, their rate limiting rules execute before traffic reaches your servers. No application-layer latency, no extra dependency to operate. The constraint: you can't vary limits by user tier without extra config work, and business-logic-aware rules require the Workers layer. Edge limits work well for coarse protection against bots and unauthenticated traffic. Per-user-tier limits are easier to own in-app.

**Approximate counting cuts Redis load at high throughput.** Keep a local counter and sync to Redis every N requests. You'll overshoot the limit slightly during the sync window, but you reduce Redis calls by 10-100x. Consider this when the round trip shows up in your traces and exact accuracy matters less than throughput.

Why This Failure Mode Is So Common

Per-instance rate limiting passes every test you'd write in development. One server, one counter, results are predictable. The failure only surfaces when you add instances, and it fails silently. No error, no alert. Users just get more headroom than you intended.

The root issue is that rate limiting is state that spans your whole fleet. Once you frame it that way, the solution follows directly: put the state somewhere the whole fleet can reach it, and make the read-modify-write a single atomic operation.

Building it with Redis from the start costs one round trip per request. Building it in-process and then retrofitting costs an incident and a postmortem.