Design Distributed Cache

Problem Statement

Design A Distributed Cache System

OR

Build a distributed in-memory caching system like Redis or Memcached.

Full Question:

Design a distributed in-memory caching system similar to Redis with the following features:

Clarifying Questions to Ask?

• What is the primary use case for the distributed cache?

• What data consistency is required?

• What is the expected access pattern?

• What are the failure and recovery expectations?

• Is persistence required in the cache?

Core Requirements:

• Key-value storage with Time-To-Live (TTL) support.
• Support for multiple data types (strings, lists, sets, hashes).
• Distributed architecture with data sharding across multiple nodes.
• Replication for high availability.
• Cache eviction policies (e.g., LRU, LFU).

Scale Requirements:

• Handle 1 million requests per second (RPS).
• Store 1TB of data across the cluster.
• 99.99% availability.
• Sub-millisecond response times for get/set operations.

Additional Considerations:

• How would you distribute data across nodes? (e.g., Consistent Hashing).
• How would you handle cache consistency with the primary database?
• How would you handle node failures and rebalancing?
• How would you mitigate the 'cache stampede' (thundering herd) problem?