Skip to content

Should I Cache?

1. Decision

Introduce caching only when read volume significantly exceeds write volume (Read-to-Write ratio > 10:1), latency SLAs cannot be met by database query tuning/indexing alone, or downstream database resources need protection from read thrashing.

2. 30-Second Answer

Caching is an optimization strategy, not a default requirement. Introduce a cache when data is read frequently, written infrequently, and tolerant of transient stale reads. Caching trades memory footprint and architectural complexity (cache invalidation, stale reads, dual-write races, cache stampedes) for microsecond read latencies and database CPU offloading. If data changes constantly (Write-heavy), reads are strictly unique per request, or strong transactional consistency is required on every read, adding a cache introduces unnecessary operational overhead and bugs without delivering performance gains.

3. Decision Matrix

Option Best For Avoid When Latency Scalability Consistency Operational Complexity
Add Cache Layer (Redis/Memcached) High read-to-write ratio (>10:1), slow expensive queries, high traffic static metadata High write-to-read ratio (<2:1), strict strong read consistency, unique single-use queries Sub-millisecond (<1ms) Scale-out read capacity instantly Eventual (Stale read risks until TTL/invalidation) High (Cache invalidation, stampede, memory tuning)
No Cache (Database Only) Strong real-time consistency requirements, financial transaction ledgers, write-heavy datasets Slow complex queries exceeding DB SLAs under load Low to Medium (2-20ms DB page hits) Database scale-up; Read Replicas Strong ACID Low

4. Selection Rules

Requirement Best Choice Rationale / Example
Read-Heavy Static Workload (95% Reads) Add Cache Product details, user profile static fields, app config flags
Write-Heavy / Streaming Telemetry No Cache High-velocity IoT sensor metrics, live location tracking updates
Expensive Aggregation / Join Queries Add Cache Pre-calculated dashboard summary analytics or leaderboard rankings
Strict Real-Time Account Balance Verification No Cache Reading stale cache state risks overdrafts or double-spending
Low-Traffic Application (<50 RPS) No Cache Database indexes handle traffic comfortably; cache adds premature complexity
Protect Database from Traffic Spikes Add Cache Offload peak traffic during flash sales / breaking news events

5. Tradeoffs

  1. Microsecond Read Latency vs Cache Invalidation Complexity
  2. Gain: Sub-millisecond responses without touching database disk or CPU.
  3. Lose: Managing cache invalidation bugs (Phil Karlton: "Hardest problem in CS").
  4. Database Load Protection vs Memory Cost Overhead
  5. Gain: Offloads up to 99% of read queries away from primary storage engines.
  6. Lose: In-memory RAM cost (Redis clusters) adds significant infrastructure billing.
  7. High System Availability vs Stale Read Risks
  8. Gain: Serves cached responses even when primary database experiences transient outages.
  9. Lose: Clients may read outdated data during invalidation lag windows.
  10. Read Performance Boost vs Write Amplification Penalty
  11. Gain: Accelerated read throughput for popular hot keys.
  12. Lose: Every database write now requires invalidating/updating cache keys.
  13. Architectural Simplicity vs Multi-Tier Failure Modes
  14. Gain: Scalable sub-millisecond response SLAs.
  15. Lose: Vulnerable to cache stampedes, penetration, and cold-start DB crashes.

6. Interview Counter Questions

  1. When is adding a cache an architectural anti-pattern?
  2. What is the Read-to-Write ratio threshold where caching becomes ineffective or detrimental?
  3. How do you quantify whether to optimize a slow query in the database vs adding a Redis cache?
  4. What happens to the primary database when a cache node with 99% hit rate suddenly crashes (Cold Cache)?
  5. What is Cache Penetration and how do Bloom Filters prevent it?
  6. How do you handle caching when data requires strict row-level authorization per user?
  7. What is Cache Breakdown vs Cache Avalanche vs Cache Stampede?
  8. How does caching affect data consistency in a distributed microservices environment?
  9. When would you choose database read-replicas over an in-memory Redis cache?
  10. How do you measure Cache Hit Ratio and what is an acceptable production threshold?

7. Gold Standard Answers

  1. Caching is an anti-pattern when applied to write-heavy datasets, unique single-use search queries, strict real-time transactional ledgers, or low-traffic systems where proper database indexing would solve the latency issue cleanly.
  2. If Read-to-Write ratio is under 3:1, caching adds write invalidation overhead and cache misses dominate, consuming RAM without delivering latency benefits.
  3. Always optimize DB indexes, query plans, and schema first. If the DB achieves \(<10\text{ms}\) p99 at peak target QPS, omit the cache. Add a cache only when complex joins/aggregations exceed SLA or DB CPU limits are reached.
  4. Cold Cache failure: Traffic bypasses the dead cache, slamming the database directly with 100% of read traffic, causing database connection pool exhaustion and cascading system outage.
  5. Cache Penetration occurs when queries search for non-existent keys (e.g., random IDs), bypassing cache and hitting the DB repeatedly. Use a Bloom Filter in front of the cache or cache NULL values with short TTLs.
  6. Either cache raw entity data un-transformed and execute authorization filtering in the application layer, or incorporate tenant/user security scope directly into cache key names (cache:user:122:document:456).
  7. Cache Breakdown = Single hot key expires \(\rightarrow\) traffic slams DB. Cache Avalanche = Millions of keys expire at the exact same timestamp \(\rightarrow\) DB crashes. Cache Stampede = Concurrent threads compute the exact same expensive missing key simultaneously.
  8. Caching creates eventual consistency. Microservices reading local or distributed caches will observe state lag relative to the primary database system of record until invalidation completes.
  9. Choose read-replicas when queries require complex SQL joins, filtering, and pagination over large datasets that don't map cleanly into key-value data structures.
  10. Measure Cache Hit Ratio = \(\frac{\text{Cache Hits}}{\text{Cache Hits} + \text{Cache Misses}}\). Target \(>90-95\%\) for production read-heavy workloads. A ratio \(<70\%\) indicates wrong key selection or inadequate TTL settings.

8. Common Mistakes

  • Adding a cache as a band-aid for missing database indexes or poorly written SQL queries.
  • Caching rapidly changing state (e.g., live stock ticker or GPS coordinates), forcing non-stop cache invalidations.
  • Failing to load-test system behavior when the cache is completely empty or unavailable (Cold Start failure).
  • Forgetting to set a TTL on dynamic cache keys, causing Redis memory exhaustion (OOM).
  • Caching sensitive user data without security scope in key names, resulting in unauthorized cross-tenant data leakage.

9. Keywords

Caching, Read-to-Write Ratio, Cold Cache, Cache Penetration, Bloom Filter, Cache Breakdown, Cache Avalanche, Cache Stampede, Cache Hit Ratio, TTL, Eventual Consistency, Phil Karlton.

10. Cheat Sheet

  • Golden Rule: Caching is an optimization, not a requirement. Optimize DB indexes first!
  • Read-to-Write Ratio: Cache if Read:Write \(> 10:1\). Avoid if Read:Write \(< 3:1\).
  • When NOT to Cache: Write-heavy streams, financial transactions, low traffic (<50 RPS), unique ad-hoc queries.
  • Cache Hit Target: \(>90-95\%\) hit ratio in production. \(<70\%\) indicates misconfiguration.
  • Cold Start Risk: Always implement rate limiting, circuit breakers, and warm-up scripts for cache restarts.
  • Security Rule: Include tenant/user ID in cache key strings if data requires authorization boundaries.

11. Related Decisions