Your In-Memory Rate Limiter Doesn't Survive the Second Pod

The Setup That Feels Right

You're building an API. Someone says "we need rate limiting." You reach for middleware:

const counts = new Map();

function rateLimit(userId, limit = 100) {
  const windowStart = Date.now() - 60_000;
  const recent = (counts.get(userId) ?? []).filter(t => t > windowStart);
  recent.push(Date.now());
  counts.set(userId, recent);
  return recent.length <= limit;
}

This passes your unit tests. Works fine in staging. You ship it.

Then you scale to three pods and your rate limiter quietly allows 3× the intended traffic.

Why It Breaks Silently

The dangerous thing isn't that this crashes — it's that it *doesn't*. Each pod has its own Map. Each pod counts its own slice of requests. With a round-robin load balancer, a user sending 300 requests per minute lands roughly 100 on each pod. Each pod says "looks fine." Nobody gets rate limited.

Worse: these counters live in process memory, so they reset on every deploy. A user who actually hits a limit just waits for the next rolling restart. You shipped security theater that fails in both directions — too permissive under normal load, too strict after restarts.

The Standard Fix: Shared Redis

Move your counters into Redis, which all pods share:

async function rateLimit(userId, limit = 100, windowMs = 60_000) {
  const key = `rl:${userId}`;
  const [count] = await redis
    .pipeline()
    .incr(key)
    .expire(key, Math.ceil(windowMs / 1000), 'NX')
    .exec();
  return count <= limit;
}

The NX flag sets the TTL only on first write, avoiding the race condition between INCR and EXPIRE. This is good enough for a lot of use cases.

But it's a **fixed window**. The window resets at a hard boundary. If a user sends 100 requests at 11:59:59 and 100 more at 12:00:01, both windows see exactly 100 and neither triggers. Two hundred requests in two seconds — no rate limiting. This is the kind of edge case an attacker (or a runaway client) will find.

Sliding Window With Lua

A real sliding window counts requests in the last N seconds regardless of clock boundaries. Redis sorted sets make this clean — store each request timestamp as both the score and the member:

-- atomic Lua script: KEYS[1]=key, ARGV[1]=now(ms), ARGV[2]=window(ms), ARGV[3]=limit
local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])

redis.call('ZREMRANGEBYSCORE', key, '-inf', now - window)
local count = redis.call('ZCARD', key)

if count < limit then
  redis.call('ZADD', key, now, now)
  redis.call('PEXPIRE', key, window)
  return 1
end
return 0

The entire thing runs atomically — Redis executes Lua scripts as a single transaction. No race between "check count" and "add entry."

The tradeoff: each sorted set entry costs about 65 bytes. At 100 req/min × 10k users you're looking at ~65 MB — manageable. At 100k users it starts mattering. The **sliding window counter** approximation (two fixed-window counts weighted by position in the window) gives you ~1% accuracy at 1/100th the memory. Worth knowing about if you're operating at scale.

The New SPOF You Just Created

Every incoming request now blocks on a Redis roundtrip: 1–3ms on a local network, up to 20ms cross-region. More importantly, Redis is now a new failure mode.

If Redis goes down, do you **fail open** (allow all traffic) or **fail closed** (block all traffic)?

• Fail open: rate limiting becomes best-effort. Someone can burst during outages.

• Fail closed: your API is down because your rate limiter's dependency is down.

For almost every API, fail open is the correct default. Rate limiting is abuse mitigation, not a hard security boundary. If a user bursts during a 30-second Redis outage, you detect it and handle it. Taking down your entire API because Redis hiccuped is a worse outcome by a wide margin.

try {
  const allowed = await checkRateLimit(userId);
  if (!allowed) return res.status(429).json({ error: 'Too Many Requests' });
} catch (err) {
  logger.warn({ err }, 'rate limit check failed, failing open');
  // don't block — keep the request moving
}

Keep a separate alert on Redis availability so you're not blind when this happens.

Push It to the Gateway

If you're behind a CDN or API gateway, you can skip the Redis roundtrip entirely for flood protection. Cloudflare Workers, Kong, nginx-ingress, and AWS API Gateway all have native rate limiting. The limit enforces at the edge before traffic hits your origin — no latency penalty in your app, no Redis to operate.

The catch: gateway rate limiting is per-IP or per-API-key using approximate algorithms (usually leaky bucket or GCRA). It can't apply business logic like "this customer's plan allows 10k calls/month." So production APIs end up using both layers: gateway limits for traffic floods, Redis for per-user plan enforcement. That's the right architecture for anything under real load.

What to Actually Ship

• **Single instance / dev**: in-memory is fine for local testing. Don't promote it.

• **Multi-pod, traffic is modest**: Redis INCR + EXPIRE with fail-open. Simple, covers 90% of cases.

• **Multi-pod, accuracy matters**: Redis sorted set + Lua sliding window. Sliding window counter hybrid if memory is a concern at scale.

• **Behind a CDN**: gateway rate limiting for flood protection, Redis for plan-level limits. Both, layered.

• **Cross-region**: per-region Redis with approximate global counts, or accept that exact global rate limiting requires cross-region synchronization you almost certainly don't need.

One thing that's non-negotiable regardless of approach: put X-RateLimit-Remaining and Retry-After headers on every response. Even if your algorithm has some imprecision, clients can see what's happening and back off gracefully. Without headers, a well-behaved client that hits a 429 will retry immediately — and make the problem worse.