Distributed Cache Systems
Distributed Cache Systems¶
Overview¶
This document covers core concepts of distributed cache systems. Learn about Redis data structures and cluster configuration, comparison with Memcached, and Consistent Hashing.
Difficulty: βββ Estimated Learning Time: 2-3 hours Prerequisites: 06_Caching_Strategies.md
Table of Contents¶
- What is Distributed Cache?
- Redis Data Structures
- Redis Cluster and Sentinel
- Memcached Comparison
- Consistent Hashing
- Practice Problems
- Next Steps
- References
1. What is Distributed Cache?¶
1.1 Need for Distributed Cache¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Need for Distributed Cache β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Local cache problems: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β [Server 1] [Server 2] [Server 3] β β
β β Local Cache Local Cache Local Cache β β
β β user:123 β user:123 β user:123 β β β
β β β β
β β Problems: β β
β β 1. Different cache on each server (inconsistency) β β
β β 2. Memory waste (duplicate data) β β
β β 3. Difficult cache invalidation β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β With distributed cache: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β [Server 1] [Server 2] [Server 3] β β
β β β β β β β
β β ββββββββββββββββΌβββββββββββββββ β β
β β β β β
β β βΌ β β
β β ββββββββββββββββββββββ β β
β β β Distributed β β β
β β β Cache (Redis) β β β
β β β user:123 β β β β
β β ββββββββββββββββββββββ β β
β β β β
β β Benefits: β β
β β 1. All servers share same cache β β
β β 2. Cache consistency maintained β β
β β 3. Centralized management β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
1.2 Local Cache vs Distributed Cache¶
| Item | Local Cache | Distributed Cache |
|---|---|---|
| Storage location | Application memory | Separate server (Redis etc.) |
| Speed | Very fast (ns~us) | Fast (ms) |
| Capacity | Server memory limit | Scalable |
| Consistency | Different per server | Shared/consistent |
| Failure | Lost on server restart | Can be managed independently |
| Use case | Single server, read-only | Multiple servers, sessions etc. |
2. Redis Data Structures¶
2.1 String¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β String β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Most basic key-value storage β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β SET user:123:name "John" β β
β β GET user:123:name β "John" β β
β β β β
β β # Set expiry time (seconds) β β
β β SET session:abc "data" EX 3600 β β
β β SETEX session:abc 3600 "data" # Same β β
β β β β
β β # Atomic increment/decrement β β
β β SET counter 0 β β
β β INCR counter β 1 β β
β β INCRBY counter 10 β 11 β β
β β DECR counter β 10 β β
β β β β
β β # NX (Not eXists): Set only if key doesn't exist β β
β β SET lock:resource "owner" NX EX 30 # Distributed lock β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Use cases: β
β β’ Session storage β
β β’ Cache (JSON serialization) β
β β’ Counters (views, likes) β
β β’ Distributed locks β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
2.2 Hash¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Hash β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Collection of field-value pairs (good for storing objects) β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β # Store user object β β
β β HSET user:123 name "John" age 30 email "john@example.com" β β
β β β β
β β # Get single field β β
β β HGET user:123 name β "John" β β
β β β β
β β # Get all fields β β
β β HGETALL user:123 β β
β β β {"name": "John", "age": "30", "email": "..."} β β
β β β β
β β # Increment field β β
β β HINCRBY user:123 age 1 β 31 β β
β β β β
β β # Check field existence β β
β β HEXISTS user:123 name β 1 β β
β β β β
β β Structure: β β
β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β
β β β user:123 β β β
β β β βββ name: "John" β β β
β β β βββ age: 30 β β β
β β β βββ email: "john@example.com" β β β
β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Advantages (vs String + JSON): β
β β’ Access/modify individual fields β
β β’ Efficient partial updates β
β β’ Memory efficient (for small hashes) β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
2.3 List¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β List β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Ordered collection of strings (Linked List) β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β # Push to right (Queue use) β β
β β RPUSH queue:jobs "job1" "job2" "job3" β β
β β β β
β β # Pop from left β β
β β LPOP queue:jobs β "job1" β β
β β β β
β β # Push to left (Stack use) β β
β β LPUSH stack:items "item1" β β
β β LPOP stack:items β "item1" β β
β β β β
β β # Range query β β
β β LRANGE queue:jobs 0 -1 β ["job2", "job3"] β β
β β β β
β β # Blocking Pop (for job queue) β β
β β BLPOP queue:jobs 30 # Wait 30 seconds β β
β β β β
β β Structure: β β
β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β
β β β queue:jobs β β β
β β β [job2] ββ [job3] β β β
β β β Head Tail β β β
β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Use cases: β
β β’ Job Queue β
β β’ Recent items list (recently viewed products) β
β β’ Timeline β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
2.4 Set¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Set β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Unordered collection of unique strings β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β # Add members β β
β β SADD user:123:followers "user:456" "user:789" β β
β β β β
β β # Check membership β β
β β SISMEMBER user:123:followers "user:456" β 1 β β
β β β β
β β # Get all members β β
β β SMEMBERS user:123:followers β {"user:456", "user:789"} β β
β β β β
β β # Set operations β β
β β SINTER user:123:followers user:456:followers # Intersectionβ β
β β SUNION user:123:followers user:456:followers # Union β β
β β SDIFF user:123:followers user:456:followers # Differenceβ β
β β β β
β β # Count β β
β β SCARD user:123:followers β 2 β β
β β β β
β β Structure: β β
β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β
β β β user:123:followers β β β
β β β { "user:456", "user:789" } β β β
β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Use cases: β
β β’ Tags β
β β’ Followers/Following β
β β’ Likes user list β
β β’ Unique visitors β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
2.5 Sorted Set (ZSet)¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Sorted Set β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Set sorted by score β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β # Add with scores β β
β β ZADD leaderboard 100 "user:123" 85 "user:456" 92 "user:789"β β
β β β β
β β # Range query (ascending) β β
β β ZRANGE leaderboard 0 2 WITHSCORES β β
β β β [("user:456", 85), ("user:789", 92), ("user:123", 100)] β β
β β β β
β β # Range query (descending) - Top 3 β β
β β ZREVRANGE leaderboard 0 2 WITHSCORES β β
β β β [("user:123", 100), ("user:789", 92), ("user:456", 85)] β β
β β β β
β β # Score range query β β
β β ZRANGEBYSCORE leaderboard 80 95 WITHSCORES β β
β β β β
β β # Rank query β β
β β ZRANK leaderboard "user:123" β 2 (0-based) β β
β β ZREVRANK leaderboard "user:123" β 0 (1st place) β β
β β β β
β β # Increment score β β
β β ZINCRBY leaderboard 10 "user:456" β 95 β β
β β β β
β β Structure: β β
β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β
β β β leaderboard β β β
β β β score: 85 β "user:456" β β β
β β β score: 92 β "user:789" β β β
β β β score: 100 β "user:123" β β β
β β βββββββββββββββββββββββββββββββββββββββββββββββββββ β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Use cases: β
β β’ Leaderboard β
β β’ Priority queue β
β β’ Timeline sorted by time (score = timestamp) β
β β’ Rate Limiting (sliding window) β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
2.6 Data Structure Summary¶
| Type | Structure | Time Complexity | Use Case |
|---|---|---|---|
| String | Key-Value | O(1) | Cache, session, counter |
| Hash | Field-Value Map | O(1) | Objects, user profiles |
| List | Linked List | O(1) Push/Pop | Queue, recent items |
| Set | HashSet | O(1) | Tags, unique visitors |
| Sorted Set | Skip List | O(log N) | Leaderboard, ranking |
3. Redis Cluster and Sentinel¶
3.1 Redis Sentinel¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Redis Sentinel β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β "Monitoring and automatic failover for Redis high availability"β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β βββββββββββββββββββββββββββββββββββββββ β β
β β β Sentinels β β β
β β β βββββββββ βββββββββ βββββββββ β β β
β β β βSent 1 β βSent 2 β βSent 3 β β β β
β β β βββββ¬ββββ βββββ¬ββββ βββββ¬ββββ β β β
β β β β β β β β β
β β ββββββββΌββββββββββΌββββββββββΌββββββββββ β β
β β β β β β β
β β ββββββββ΄ββββββββββ΄ββββββββββ΄βββββββ β β
β β β β β β
β β βΌ βΌ β β
β β βββββββββββ βββββββββββ β β
β β β Master β ββββreplicationβββΆ β Replica β β β
β β β (R/W) β β (R/O) β β β
β β βββββββββββ βββββββββββ β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Roles: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β 1. Monitoring: Watch Master/Replica status β β
β β 2. Notification: Alert admins on failure β β
β β 3. Automatic failover: Promote Replica to Master on failureβ β
β β 4. Configuration provider: Provide current Master address β β
β β to clients β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Failover process: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β 1. Detect Master failure (majority Sentinel agreement) β β
β β 2. Elect one Sentinel as leader β β
β β 3. Leader promotes one Replica to Master β β
β β 4. Other Replicas connect to new Master β β
β β 5. Notify clients of new Master address β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Limitations: β
β β’ No horizontal scaling (single Master) β
β β’ Writes only to Master β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
3.2 Redis Cluster¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Redis Cluster β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β "Redis horizontal scaling and automatic sharding" β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β 16384 Hash Slots distribution: β β
β β β β
β β βββββββββββββββ βββββββββββββββ βββββββββββββββ β β
β β β Node A β β Node B β β Node C β β β
β β β Master β β Master β β Master β β β
β β β Slots: β β Slots: β β Slots: β β β
β β β 0-5460 β β 5461-10922 β β 10923-16383 β β β
β β ββββββββ¬βββββββ ββββββββ¬βββββββ ββββββββ¬βββββββ β β
β β β β β β β
β β βΌ βΌ βΌ β β
β β βββββββββββββββ βββββββββββββββ βββββββββββββββ β β
β β β Replica A β β Replica B β β Replica C β β β
β β βββββββββββββββ βββββββββββββββ βββββββββββββββ β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Hash Slot calculation: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β slot = CRC16(key) % 16384 β β
β β β β
β β Example: key = "user:123" β β
β β CRC16("user:123") = 12345 β β
β β slot = 12345 % 16384 = 12345 β Node C β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Hash Tag (force same slot): β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β # {user:123} is hash target β β
β β user:{user:123}:profile β β
β β user:{user:123}:settings β β
β β user:{user:123}:orders β β
β β β β
β β β All stored in same slot β β
β β β Multi-key operations possible β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Features: β
β β’ Horizontal scaling (add nodes) β
β β’ Automatic sharding β
β β’ High availability (auto Replica failover) β
β β’ Some limitations: Multi-key ops only within same slot β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
3.3 Sentinel vs Cluster¶
| Item | Sentinel | Cluster |
|---|---|---|
| Purpose | High availability | HA + Horizontal scaling |
| Sharding | None (single Master) | Auto sharding |
| Capacity | Single server memory | Distributed storage |
| Complexity | Low | High |
| Multi-key ops | Free | Hash Tag needed |
| Suitable for | Small-medium scale | Large scale |
4. Memcached Comparison¶
4.1 Redis vs Memcached¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Redis vs Memcached β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Redis: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β β’ Various data structures (String, Hash, List, Set, ZSet) β β
β β β’ Persistence options (RDB, AOF) β β
β β β’ Replication and cluster support β β
β β β’ Pub/Sub, Lua scripts β β
β β β’ Transactions (MULTI/EXEC) β β
β β β’ Single-threaded (event loop) β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Memcached: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β β’ Simple key-value storage (String only) β β
β β β’ No persistence (pure cache) β β
β β β’ Multi-threaded (utilizes multi-core) β β
β β β’ Simple and lightweight β β
β β β’ LRU cache policy β β
β β β’ Slab allocator (memory efficient) β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
4.2 Comparison Table¶
| Item | Redis | Memcached |
|---|---|---|
| Data structures | Various | String only |
| Persistence | RDB, AOF | None |
| Replication | Supported | None |
| Cluster | Supported | Client sharding |
| Threading | Single (6.0+ multi) | Multi |
| Memory efficiency | Good | Very good |
| Max value size | 512MB | 1MB |
| Use cases | General purpose, session, queue | Simple cache |
4.3 Selection Criteria¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Selection Guide β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Choose Redis when: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β’ Need complex data structures (List, Set, Sorted Set) β β
β β β’ Need data persistence β β
β β β’ Need Pub/Sub, message queue features β β
β β β’ Need replication and high availability β β
β β β’ Session storage β β
β β β’ Leaderboard, Rate Limiting β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Choose Memcached when: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β’ Only need simple key-value cache β β
β β β’ Multi-core utilization important β β
β β β’ Memory efficiency critical β β
β β β’ Persistence not needed β β
β β β’ Very high throughput needed β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Most cases recommend Redis (feature diversity, ecosystem) β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
5. Consistent Hashing¶
5.1 Traditional Hashing Problem¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Traditional Hashing Problem β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Modular hashing: hash(key) % N β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β 3 servers: β β
β β hash("user:1") % 3 = 1 β Server 1 β β
β β hash("user:2") % 3 = 2 β Server 2 β β
β β hash("user:3") % 3 = 0 β Server 0 β β
β β β β
β β Add 1 server (4 total): β β
β β hash("user:1") % 4 = 0 β Server 0 (changed!) β β
β β hash("user:2") % 4 = 2 β Server 2 β β
β β hash("user:3") % 4 = 3 β Server 3 (changed!) β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Problem: Most keys redistributed on server add/remove! β
β β Cache miss surge β Database load β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β N servers β N+1 servers: N/(N+1) keys redistributed β β
β β Example: 100 β 101 servers: ~99% keys redistributed! β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
5.2 Consistent Hashing Principle¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Consistent Hashing β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β "Minimize key redistribution on server add/remove" β
β β
β Hash Ring: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β 0 β β
β β β β β
β β ββββββ΄βββββ β β
β β β β β β
β β Node A βββ βββ key1 β β
β β / \ β β
β β / \ β β
β β key2 βββββ ββββ Node B β β
β β Node C \ β β
β β \ \ β β
β β \ βββ key3 β β
β β \ / β β
β β βββββββββββββββ β β
β β β β
β β Key β Assigned to first node clockwise β β
β β β β
β β key1 β Node B β β
β β key2 β Node C β β
β β key3 β Node A β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β When adding node: β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β Node A βββ β β
β β / β β
β β key2 βββββ βββ Node D (newly added)β β
β β Node C β β β
β β βββ key1 (moved to D) β β
β β β β β
β β ββββββββββββββββββ β β
β β Node B β β
β β β β
β β Only key1 moved: Node B β Node D β β
β β Others stay! β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Theory: Only K/N keys redistributed (K=total keys, N=nodes) β
β Example: 100 nodes β 101: Only ~1% redistributed! β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
5.3 Virtual Nodes¶
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Virtual Nodes β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β Problem: Uneven distribution with few nodes β
β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β With only 2 nodes: β β
β β β β
β β Node A βββ β β
β β / \ β β
β β / \ β β
β β / \ β β
β β / Many keys β β
β β / go to β β
β β ββββββ Node B β β
β β β β
β β β Uneven! β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Solution: Use virtual nodes β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β β β
β β Represent each physical node as multiple virtual nodes β β
β β β β
β β Node A β A-1, A-2, A-3, A-4, ... β β
β β Node B β B-1, B-2, B-3, B-4, ... β β
β β β β
β β A-1 β β β
β β / B-2 β β β
β β / / A-3 β β β
β β B-1 β / / β β
β β / / β β
β β A-2 βββββββββ B-3 β β
β β β β
β β β More even distribution! β β
β β β β
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β
β Virtual node count: β
β β’ More nodes = more even, but higher memory usage β
β β’ Typically 100-200 virtual nodes per physical node β
β β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
6. Practice Problems¶
Problem 1: Redis Data Structure Selection¶
Choose the appropriate Redis data structure for the following requirements:
a) Store user session (ID, name, permissions, etc.) b) Real-time game leaderboard c) Store 10 recently viewed products d) List of posts user liked e) Chat room message queue
Problem 2: Sentinel vs Cluster¶
Choose between Sentinel and Cluster for the following scenarios:
a) 10GB data, high availability needed b) 1TB data, horizontal scaling needed c) Service with many complex transactions d) Simple cache, 1 million requests per second
Problem 3: Consistent Hashing¶
With 4 cache servers, when adding 1 server: a) What % of keys redistributed with traditional hashing? b) What % of keys redistributed with consistent hashing?
Problem 4: Redis Design¶
Design a social media follow feature using Redis.
Requirements: - User A follows B - View A's following list - View B's follower list - Find common following between A and B
Answers¶
Problem 1 Answer¶
a) User session: Hash
HSET session:abc123 user_id 123 name "John" role "admin"
b) Leaderboard: Sorted Set
ZADD leaderboard 1000 "user:123" 950 "user:456"
c) Recently viewed: List (with size limit)
LPUSH user:123:recent "product:789"
LTRIM user:123:recent 0 9 # Keep only 10
d) Liked posts: Set
SADD user:123:likes "post:456" "post:789"
e) Chat message queue: List
RPUSH chat:room1 "{message...}"
BLPOP chat:room1 0 # Consumer waits
Problem 2 Answer¶
a) 10GB, HA: Sentinel
- Data size fits single server
- Sentinel provides HA
- Simple configuration
b) 1TB, horizontal scaling: Cluster
- Large data needs distributed storage
- Horizontal scaling for throughput
c) Complex transactions: Sentinel
- Cluster has multi-key transaction limitations
- Single Master easier for transactions
d) Simple cache, high throughput: Cluster or Memcached
- Cluster for load distribution
- Memcached also viable for simple cache
Problem 3 Answer¶
a) Traditional hashing: ~80%
N/(N+1) = 4/5 = 80% keys redistributed
b) Consistent hashing: ~20%
K/N = 1/5 = 20% keys only
(keys in new node's range)
Problem 4 Answer¶
# Follow relationship
# A follows B
SADD user:A:following "B"
SADD user:B:followers "A"
# A's following list
SMEMBERS user:A:following
# B's follower list
SMEMBERS user:B:followers
# Common following between A and B
SINTER user:A:following user:B:following
# Follower count
SCARD user:B:followers
# Check if following
SISMEMBER user:A:following "B"
# Unfollow
SREM user:A:following "B"
SREM user:B:followers "A"
7. Next Steps¶
If you've understood distributed cache, learn about database scaling next.
Next Lesson¶
Related Lessons¶
- 06_Caching_Strategies.md - Caching patterns
- 09_Database_Replication.md - Replication strategies
Recommended Practice¶
- Install Redis and practice data structures
- Configure Redis Sentinel
- Implement consistent hashing yourself
8. References¶
Official Documentation¶
Tools¶
- Redis Commander - GUI
- RedisInsight - Official GUI
Papers¶
- "Consistent Hashing and Random Trees" - Karger et al.
Document Information - Last Modified: 2024 - Difficulty: βββ - Estimated Learning Time: 2-3 hours