Consistent Hashing: How CDNs and Caches Scale
Issue #60 : System Design Interview Roadmap
The Hidden Architecture Behind Every Web Request
When you click a video on YouTube, stream music on Spotify, or browse Instagram photos, you're witnessing one of distributed systems' most elegant solutions in action. Behind the scenes, consistent hashing quietly orchestrates how millions of requests find their way to the right servers, ensuring your content loads instantly from the nearest cache.
Unlike traditional hashing where adding a single server can trigger a complete data reshuffling nightmare, consistent hashing achieves something remarkable: it can add or remove nodes with minimal data movement, typically affecting only 1/N of the total dataset where N is the number of nodes. This seemingly simple property has become the backbone of every major distributed cache, CDN, and NoSQL database you've ever used.
The Traditional Hashing Problem That Breaks at Scale
Most engineers intuitively reach for modular hashing when distributing data across multiple servers: hash(key) % server_count. This approach works beautifully for static clusters, but it harbors a catastrophic flaw that becomes apparent only at scale.
Consider what happens when your traffic grows and you need to add a server to your 3-node cache cluster. Suddenly, hash(key) % 3 becomes hash(key) % 4, causing approximately 75% of your cached data to become unreachable. Every cache miss translates to a database query, potentially triggering a cascading failure that can bring down your entire system.
This isn't just a theoretical concern. Netflix learned this lesson the hard way during their early scaling phase when adding cache nodes during peak traffic hours would effectively invalidate their entire cache layer, causing their backend databases to buckle under the sudden load spike.
Consistent Hashing: The Ring That Changes Everything
Consistent hashing solves this problem through an elegant conceptual shift: instead of mapping keys directly to servers, it maps both keys and servers to points on a circular hash space, typically a 32-bit or 64-bit ring.
