Skip to content

Circuit Breaker

1. Decision

Implement the Circuit Breaker Pattern (Resilience4j, Envoy, Istio) on all inter-service network boundaries to detect failing downstream dependencies, fail fast immediately without waiting for timeouts, and isolate microservice thread pools from cascading outages.

2. 30-Second Answer

When a downstream microservice or database experiences high latency or outages, calling services continue sending requests. Incoming threads block waiting for timeouts, filling thread pools (Tomcat/Netty) and causing Cascading Failures that crash the entire upstream application. The Circuit Breaker Pattern acts as an automatic safety switch monitoring downstream calls across three states: 1. Closed (Normal operation): Requests pass through; failure metrics are tracked in a sliding window. 2. Open (Tripped / Failure threshold exceeded): Incoming requests fail immediately without calling the downstream dependency, returning a fast fallback error (503 / default payload). 3. Half-Open (Trial recovery): Allows a small sample of test requests through to check if the downstream service has recovered.

3. Decision Matrix

Option Best For Avoid When Latency Scalability Consistency Operational Complexity
Circuit Breaker (Resilience4j / Envoy) Microservice-to-microservice RPCs, external third-party API integration, database calls, preventing thread pool exhaustion In-process local method calls, single monolithic applications without network boundaries Low (Sub-millisecond state check; zero network latency when Open) High (Prevents thread pool exhaustion & cascading cluster outages) Fail-fast error or Fallback data payload Medium
Simple Timeout Only (No Circuit Breaker) Standalone single API calls, low-volume background batch scripts Distributed microservice call chains (A \(\rightarrow\) B \(\rightarrow\) C) under high traffic volume Bounded by Timeout value (e.g. 2,000ms per failed call) Poor under sustained downstream outages (Thread pool fills up) Standard error return Low

4. Selection Rules

Requirement Best Choice Rationale / Example
Isolate Upstream Services from Downstream Outages Circuit Breaker Trip circuit when Payment Service error rate exceeds 50%, failing fast immediately
Prevent Thread Pool Exhaustion Across Microservices Circuit Breaker + Bulkhead Open circuit prevents 500 API Gateway threads from blocking on a 10s database timeout
Serve Degradation Fallbacks During Outages Circuit Breaker + Fallback Return cached recommendations or empty array ([]) when Recommendation API is Open
Offload Circuit Breaking to Service Mesh Envoy / Istio Outlier Detection Envoy sidecar automatically ejects failing pod endpoints from load balancing pool

5. Tradeoffs

  1. Fail-Fast System Protection vs Temporary Request Rejection
  2. Gain: Immediately frees worker threads; isolates failing microservices from crashing upstream caller.
  3. Lose: Rejects requests immediately during the Open window even if a specific downstream instance recovers.
  4. Graceful Fallback Data vs Complete Error Failure
  5. Gain: Users receive partial page UI rendering (cached data / default values) instead of blank 500 error pages.
  6. Lose: Requires writing and testing fallback logic for every integrated domain endpoint.
  7. Sliding Window Failure Precision vs Memory/CPU Overhead
  8. Gain: Accurately trips based on error percentage (e.g., \(>50\%\) failure rate over last 100 calls) or slow calls.
  9. Lose: Maintains in-memory ring buffers tracking recent call execution statistics.
  10. Half-Open Automatic Recovery vs Flapping Instability
  11. Gain: Automatically restores traffic flow once downstream service recovers health.
  12. Lose: Poorly tuned Half-Open trial windows cause the breaker to flap repeatedly between Open and Closed.
  13. App Library (Resilience4j) vs Service Mesh (Envoy Outlier Detection)
  14. Gain (Mesh): Managed centrally in infrastructure without language-specific code modifications.
  15. Lose (Mesh): Cannot execute domain-specific application Fallback functions (returns raw Envoy 503).

6. Interview Counter Questions

  1. Explain the 3 states of a Circuit Breaker (Closed, Open, Half-Open) and the transition triggers between them.
  2. What is the difference between Count-Based Sliding Window and Time-Based Sliding Window in Resilience4j?
  3. How does Circuit Breaking interact with Retries and Timeouts (in what order should they be layered)?
  4. What is Outlier Detection in Envoy Proxy / Istio Service Mesh?
  5. How do you implement a Fallback function when a Circuit Breaker opens?
  6. What is Circuit Breaker Flapping and how do you prevent it?
  7. What is the difference between a Rate Limiter, a Bulkhead, and a Circuit Breaker?
  8. How do slow calls (high latency) trigger a Circuit Breaker compared to hard 5xx exception errors?
  9. Where should Circuit Breakers be positioned in an Architecture (Edge Gateway vs Internal Microservices)?
  10. What metrics must be monitored on Circuit Breakers in production (Prometheus/Grafana)?

7. Gold Standard Answers

  1. Closed: Normal state, calls pass through. If failure rate exceeds threshold (e.g. 50%), transitions to Open. Open: All calls fail fast immediately without executing remote RPC. After waitDurationInOpenState (e.g. 30s), transitions to Half-Open. Half-Open: Permits \(N\) trial requests; if successful, resets to Closed; if failed, returns to Open.
  2. Count-Based: Evaluates failure rate over last \(N\) calls (e.g. last 100 requests). Time-Based: Evaluates failure rate over last \(N\) seconds (e.g. last 60 seconds).
  3. Layering order (Outer to Inner): Retry \(\rightarrow\) Circuit Breaker \(\rightarrow\) Rate Limiter \(\rightarrow\) Timeout \(\rightarrow\) Remote RPC. The Circuit Breaker wraps the RPC attempt; Retry wraps the Circuit Breaker to attempt alternative instances if open.
  4. Envoy Outlier Detection tracks HTTP 5xx errors per upstream host pod. If Pod A returns \(N\) consecutive 5xx errors, Envoy ejects Pod A from the load balancing pool for 30s without opening a full service breaker.
  5. Fallback executes when Breaker is Open or throws an exception: decorateSupplier(breaker, supplier).recover(throwable -> getCachedFallbackData()).
  6. Flapping occurs when a Breaker opens, transitions to Half-Open, permits 1 request that succeeds, closes, and immediately trips Open again under load. Prevent by increasing Half-Open trial sample size (e.g., require 10 consecutive successful calls).
  7. Rate Limiter: Caps incoming request volume (QPS). Bulkhead: Limits concurrent execution threads/slots per dependency. Circuit Breaker: Stops calling a dependency when failure rate is high.
  8. Configure slowCallRateThreshold (e.g. 50% of calls exceeding slowCallDurationThreshold of 2,000ms count as failures, tripping the breaker even if no 5xx errors occur).
  9. Position at all client-side outbound HTTP/gRPC boundaries: (1) Edge API Gateway calling internal services, (2) Microservice calling downstream microservices, (3) Microservice calling DB/Redis/3rd-party APIs.
  10. Monitor state (0=Closed, 1=Open, 2=Half-Open), failure_rate, slow_call_rate, buffered_calls, not_permitted_calls_total in Grafana dashboards with P1 alerts on Open transitions.

8. Common Mistakes

  • Setting no timeouts on downstream RPCs, preventing slow calls from registering as failures in the Circuit Breaker.
  • Ordering Retries inside the Circuit Breaker instead of outside, causing retries to execute against an Open breaker.
  • Setting the Half-Open trial sample size to 1, causing the breaker to flap continuously under unstable downstream recovery.
  • Failing to write Fallback handlers, causing the application to throw raw 503 exceptions to users when the breaker opens.
  • Applying Circuit Breakers to in-process local memory method calls, adding useless overhead.

9. Keywords

Circuit Breaker, Closed, Open, Half-Open, Resilience4j, Netflix Hystrix, Envoy Outlier Detection, Fail-Fast, Fallback, Sliding Window, Count-Based, Time-Based, Slow Call Threshold, Flapping, Cascading Failure.

10. Cheat Sheet

  • Core Goal: Protect caller thread pools from Cascading Failures by failing fast when downstream services are sick.
  • 3 States:
  • Closed: Calls pass. If failure rate \(>50\% \rightarrow\) Open.
  • Open: Calls fail fast immediately (0ms). Triggers Fallback. After 30s \(\rightarrow\) Half-Open.
  • Half-Open: Sends 10 trial requests. If success \(\rightarrow\) Closed; If fail \(\rightarrow\) Open.
  • Correct Layering Order: Retry \(\rightarrow\) Circuit Breaker \(\rightarrow\) Timeout \(\rightarrow\) Remote Call.
  • Fallback Rule: Always provide a Fallback (cached data, default static JSON, or empty array) when a breaker opens.
  • Slow Call Detection: Configure slowCallDurationThreshold (e.g. >2s) to trip breakers on high latency even if no 5xx errors occur.
  • Service Mesh: Envoy Outlier Detection ejects failing single pod IPs automatically without application code.

11. Related Decisions