cachingreadslatencyhot-keys
Read-Through Caching
Cache layer loads missing keys from the backing store on demand to reduce read latency.
Definition
Read-through caching serves reads from cache and automatically fetches from the source of truth on cache miss.
When To Use
- Read-heavy access patterns with repeated key lookups.
- Latency-sensitive APIs where origin calls are expensive.
- Systems with clear cache invalidation semantics.
When Not To Use
- Highly volatile data with low reuse and high invalidation churn.
- Queries that are broad scans rather than key-based fetches.
- Cases where stale reads cannot be tolerated at all.
Tradeoffs
- Reduces p95/p99 latency, but introduces cache invalidation complexity.
- Cuts origin load, but can cause thundering-herd on hot misses.
- Improves user experience, but may serve stale data under TTL strategy.
Common Failure Modes
- Hot-key miss storm overwhelms origin (cache stampede).
- Invalidation lag serves stale data beyond product tolerance.
- Uneven key popularity creates memory fragmentation and eviction churn.
Interview Framing
Use this structure when the interviewer asks for this pattern explicitly.
Discuss TTL vs explicit invalidation, dogpile protection, hot-key mitigation, and fallback behavior during cache outages.
Related Project Deep Dives
URL Shortener at Global Scale
Design a globally distributed URL shortener that handles billions of redirects per day with low latency, abuse controls, and reliable analytics.
Distributed Cache System
Design a distributed cache system like Redis or Memcached that handles millions of requests per second with sub-millisecond latency, high availability, and intelligent eviction policies.
Related Concepts
Write-Behind Caching
Buffer writes in cache first, then flush asynchronously to the database for higher write throughput.
Consistent Hashing
Distribute keys across nodes while minimizing remapped keys during node add/remove events.
Circuit Breaker
Protect services from cascading failures by short-circuiting calls to unhealthy dependencies.