Skip to content

Local Cache vs Distributed Cache

1. Decision

Choose a Local Cache (In-Memory inside application process e.g., Caffeine, Guava, Ehcache) for ultra-low latency (<100 microseconds), static read-heavy reference data bounded in size; choose a Distributed Cache (Redis, Memcached) for shared, centralized cache state accessed consistently across multiple microservice application nodes.

2. 30-Second Answer

Local Caching stores data directly inside the heap memory of individual application process instances (e.g., Caffeine in Java, LRU cache in Node/Go). It provides extreme nanosecond/microsecond read performance (\(<0.1\text{ms}\)) by eliminating network serializations and I/O sockets, but suffers from cache drift (inconsistency across instances), duplicated RAM utilization, and cold starts upon deployment restarts. Distributed Caching (e.g., Redis Cluster, Memcached) centralizes cache state in a dedicated network service accessed by all application instances. It guarantees single-source-of-truth consistency across instances and survives application deployments, but introduces network round-trip overhead (\(1-2\text{ms}\)). High-scale architectures often combine both in a Two-Tier Caching layout.

3. Decision Matrix

Option Best For Avoid When Latency Scalability Consistency Operational Complexity
Local Cache (In-App Heap) Ultra-fast access (<0.1ms), static reference tables, feature flags, hot key shielding Multi-instance consistency requirements, dynamic dynamic state, large datasets exceeding JVM heap Ultra-Low (<0.1ms nanosecond memory lookup) Bounded by single node RAM; Duplicated across N nodes Weak (Cache drift across application instances) Very Low (Library import)
Distributed Cache (Redis) Shared microservice state, user sessions, central invalidation, datasets larger than app memory Strict sub-millisecond (<0.1ms) latency requirements where network I/O is prohibited Low (1-2ms in-datacenter network RTT) Independent horizontal scale (Redis Cluster) Strong Central Consistency (Single shared cache state) Medium

4. Selection Rules

Requirement Best Choice Rationale / Example
Sub-Millisecond SLA (<100 microseconds) Local Cache Direct heap memory access bypasses TCP network serialization overhead
Consistent State Across 50 Microservices Distributed Cache All instances read and invalidate a single centralized Redis cluster
Static Configuration / Taxonomy Data Local Cache Country codes, currency exchange reference tables updated once per day
Shared User Session / Auth Tokens Distributed Cache User requests routed to any application container read identical session state
Shield Distributed Cache from Hot Keys Two-Tier (Local + Dist) Local cache absorbs millions of reads for ultra-hot keys before hitting Redis
Large Cache Footprint (>100GB) Distributed Cache Offloads cache memory out of application JVM heap to prevent GC pause spikes

5. Tradeoffs

  1. Nanosecond Speed vs Network Latency
  2. Gain (Local): Direct pointer lookup in RAM without network round-trips or serializations.
  3. Lose (Local): Higher memory consumption inside application process; risks JVM Garbage Collection (GC) pauses.
  4. Global Consistency vs Cache Drift Inconsistency
  5. Gain (Distributed): Single source of truth; updating a key in Redis reflects instantly across all instances.
  6. Lose (Distributed): Every cache lookup incurs a network TCP hop (1-2ms RTT).
  7. Deployment Survival vs Cold-Start Wipe
  8. Gain (Distributed): Restarting or auto-scaling application containers does not empty the cache.
  9. Lose (Distributed): Network partition or Redis cluster outage impacts all application instances simultaneously.
  10. Memory Footprint Efficiency
  11. Gain (Distributed): Cache item stored exactly once in centralized Redis memory.
  12. Lose (Distributed): Local caching duplicates identical items across N application instance heaps.
  13. Architectural Complexity
  14. Gain (Local): Simple in-memory library (e.g., Caffeine); zero external infrastructure dependencies.
  15. Lose (Local): Requires pub/sub bus (e.g., Redis Pub/Sub) to broadcast invalidation signals across local caches.

6. Interview Counter Questions

  1. What is Cache Drift in local caching and how do you mitigate it across 100 microservice nodes?
  2. How does a Two-Tier (L1/L2) Caching architecture work (Local L1 + Distributed L2)?
  3. Why can large in-process JVM local caches cause catastrophic Stop-The-World Garbage Collection pauses?
  4. How do you invalidate local caches in real-time when underlying database records change?
  5. What is the impact of application rolling deployments on Local Cache vs Distributed Cache performance?
  6. How does Caffeine Cache (Java) handle near-optimal cache eviction using Window TinyLFU?
  7. When would a hot key cause a Distributed Cache to fail, requiring a Local Cache circuit breaker?
  8. How does serialization overhead differ between Local in-memory object reference vs Distributed TCP byte payloads?
  9. How do you handle cache size bounds in Local Caching to prevent Out-Of-Memory (OOM) process crashes?
  10. Which caching tier should store rate-limiting counters for an API Gateway?

7. Gold Standard Answers

  1. Cache Drift occurs when Node A updates local memory while Node B serves outdated local memory. Mitigate by setting short TTLs (e.g., 5-30s) or broadcasting invalidation events over Redis Pub/Sub / Kafka to clear local memory on all nodes.
  2. L1 is a small, ultra-fast local in-process cache (Caffeine); L2 is a centralized distributed cache (Redis). Requests check L1 first (\(<0.1\text{ms}\)); on miss, check L2 (\(1\text{ms}\)); on miss, fetch from DB and populate both L1 and L2.
  3. Allocating tens of gigabytes to JVM heap for local caching increases GC root tracing scanning time, triggering multi-second Stop-The-World (STW) GC pauses that drop application health checks.
  4. Use Change Data Capture (CDC via Debezium) or DB triggers to publish invalidation topics to Kafka / Redis Pub/Sub. Each microservice instance listens to the topic and evicts the key from its local Caffeine cache.
  5. During rolling deployments, restarting containers completely wipes Local Caches, causing a surge of cold-start traffic to the database. Distributed Cache state remains intact during app container redeployments.
  6. Window TinyLFU combines a small admission window (LRU) with a Frequency Sketch (TinyLFU counting bloom filter) to achieve higher hit ratios than standard LRU while consuming minimal memory overhead for metadata.
  7. An extreme hot key (e.g., 500,000 QPS for a celebrity profile) saturates the single Redis shard network interface. Placing a local cache in front of Redis absorbs 99% of hot key reads directly inside app memory.
  8. Local caches store direct pointers to native language memory objects (zero serialization cost). Distributed caches require serializing objects into JSON/Protobuf/MsgPack byte streams over TCP, incurring CPU overhead.
  9. Enforce strict bounded cache limits (e.g., maximumSize(10000) or maximumWeight()) using eviction algorithms (LRU/W-TinyLFU) and off-heap memory stores (e.g., DirectByteBuffer / Chronicle Map) if necessary.
  10. Distributed Cache (Redis) must store rate-limiting counters (INCR + EXPIRE) to enforce global rate limits accurately across all distributed API Gateway nodes.

8. Common Mistakes

  • Building an un-bounded local HashMap in Node.js or Java, leading to memory leaks and process crashes (OOM).
  • Using Local Cache for user authentication session state across multi-instance load-balanced web servers.
  • Failing to set short TTLs or invalidation listeners on Local Caches, causing severe data inconsistency across instances.
  • Storing massive multi-gigabyte datasets inside Java JVM heap instead of using an out-of-process Distributed Cache.
  • Over-complicating architecture by deploying Redis for a low-traffic application when a simple local cache suffices.

9. Keywords

Local Cache, Distributed Cache, Caffeine, Cache Drift, Two-Tier Caching (L1/L2), Window TinyLFU, JVM Garbage Collection, Redis Pub/Sub Invalidation, Hot Key Shielding, Off-Heap Memory, Serialization Cost.

10. Cheat Sheet

  • Core Difference: Local Cache = In-app process memory (<0.1ms, unshared, risks cache drift); Distributed Cache = Remote shared network service (1-2ms, Redis, central consistency).
  • Two-Tier Pattern: L1 Local Cache (Caffeine) \(\rightarrow\) L2 Distributed Cache (Redis) \(\rightarrow\) Database.
  • Cache Invalidation: Use Redis Pub/Sub to broadcast invalidation messages to clear local L1 caches across all app nodes.
  • Hot Key Protection: Use L1 Local Cache to shield Redis shards from extreme single-key traffic spikes (>100k QPS).
  • Session State: Always store user sessions and global rate limit counters in Distributed Cache (Redis), never in Local Cache.
  • Memory Safety: Always set strict maximumSize() bounds on Local Caches to prevent Out-Of-Memory crashes.

11. Related Decisions