Skip to content

Idempotency Strategy

1. Decision

Implement explicit Idempotency Strategies across all API endpoints and asynchronous message consumers, ensuring that executing an operation multiple times with the same input produces the exact same system state and response as executing it once.

2. 30-Second Answer

In distributed systems, network timeouts, retries, and At-Least-Once message delivery make duplicate requests inevitable. Without Idempotency, retrying a payment request or order submission causes catastrophic double-charging or duplicate order fulfillment. Idempotency guarantees that \(f(f(x)) = f(x)\). Naturally idempotent HTTP methods (GET, PUT, DELETE) require proper REST semantics. Non-idempotent operations (POST) require explicit Idempotency Keys (UUIDs generated by the client) verified server-side using a two-phase check-and-insert pattern inside an in-memory store (Redis) or database UNIQUE constraint, combined with cached response payloads.

3. Decision Matrix

Option Best For Avoid When Latency Scalability Consistency Operational Complexity
HTTP Idempotency-Key Header (Redis SET NX) Financial payment APIs, checkout operations, third-party webhooks, mobile POST endpoints Read-only endpoints (GET), naturally idempotent state replacements (PUT) Ultra-Low (<1ms Redis key check + TTL) High (Stateless API gateway / app tier with Redis cache) Strict (Prevents duplicate executions globally) Low to Medium
Database Unique Constraint (UNIQUE INDEX) Relational database entity creations, user registration, account ledger inserts Ultra-high QPS logging where index maintenance degrades write performance Low (Enforced at storage engine layer during write) Scale-up (SQL); Local shard scale-out Strong (Database engine guarantees atomicity) Low
Naturally Idempotent REST Verbs (PUT / DELETE) State replacement operations (PUT /users/123), resource deletions (DELETE /orders/456) Incremental updates (balance = balance + 100), append-only list operations Low High Strong Very Low

4. Selection Rules

Requirement Best Choice Rationale / Example
Financial Payment API Checkout Endpoint Idempotency-Key Header Client sends UUID header; server deduplicates concurrent retries via Redis
Database Row Creation Safety UNIQUE Constraint Add UNIQUE(tenant_id, idempotency_key) to database table schema
Asynchronous Message Queue Consumer (Kafka/SQS) Message Deduplication Table Check message_id against Redis/DB before executing background task
State Overwrite Operation (user.email = X) HTTP PUT Setting a value to X 10 times results in identical state X
Incremental State Update (counter += 1) Idempotency Key + Lock Non-idempotent operation requires explicit deduplication tracking

5. Tradeoffs

  1. Duplicate Execution Safety vs Memory Footprint Overhead
  2. Gain: Completely prevents double-charging, duplicate orders, and state corruption.
  3. Lose: Must store Idempotency Keys and cached response payloads in Redis/DB for \(N\) days.
  4. Client-Generated UUIDs vs Server-Generated Hash Identifiers
  5. Gain (Client UUID): Captures client-intent precisely even if payload attributes change slightly.
  6. Lose (Client UUID): Requires client SDKs to generate and persist UUIDs across network retries.
  7. Database Unique Constraints vs Redis Key Pre-Checking
  8. Gain (DB Constraint): Atomic ACID safety guaranteed by database storage engine.
  9. Lose (DB Constraint): Aborted duplicate inserts generate DB error logs and consume auto-increment IDs.
  10. Caching Response Payloads vs Storage Cost
  11. Gain: Retried requests receive exact identical HTTP response payload instantly without re-processing.
  12. Lose: Increased RAM/disk storage required to cache JSON response strings alongside keys.
  13. Short Key TTL vs Long Key TTL
  14. Gain (Long TTL): Protects against late retries days after initial failure.
  15. Lose (Long TTL): Accumulates millions of keys in Redis memory.

6. Interview Counter Questions

  1. Which HTTP methods are naturally idempotent according to RFC 7231, and why is POST not idempotent?
  2. How do you design a complete Idempotent API endpoint for credit card processing (Stripe-style)?
  3. What happens if a duplicate request arrives while the initial request is still actively processing?
  4. How do you implement Idempotency in an asynchronous Kafka or SQS consumer worker?
  5. What is the difference between Idempotent Delivery and Idempotent Processing?
  6. Why is storing the Idempotency Key in Redis after DB processing starts vulnerable to race conditions?
  7. How do you handle response payload caching for idempotent endpoints?
  8. What is the Idempotency-Key HTTP header specification drafted by the IETF?
  9. Why should Idempotency Keys have a mandatory Time-To-Live (TTL e.g. 24-72 hours)?
  10. How do you handle idempotency when an operation involves third-party downstream APIs?

7. Gold Standard Answers

  1. GET, HEAD, OPTIONS, TRACE (Safe & Idempotent); PUT, DELETE (Idempotent - state replacement); POST, PATCH (NOT Idempotent - creates resources or appends state).
  2. Client sends Idempotency-Key: <UUID>. Server checks Redis: (1) Key exists & processing \(\rightarrow\) return 409 Conflict or wait; (2) Key exists & completed \(\rightarrow\) return cached response; (3) Key new \(\rightarrow\) save key status=IN_PROGRESS and process.
  3. Use a 3-state key in Redis (IN_PROGRESS, COMPLETED, FAILED). If a duplicate request arrives while IN_PROGRESS, return 409 Conflict or block/wait until COMPLETED to return cached response.
  4. Worker extracts message_id or business payload hash. In a single local DB transaction: INSERT INTO processed_messages (message_id) VALUES (...). If duplicate, DB raises unique violation; worker skips processing and ACKs message.
  5. Idempotent Delivery (Broker setting) prevents duplicate message publishing to topic. Idempotent Processing (Consumer logic) guarantees consumer handles re-delivered duplicate messages safely.
  6. Check-then-act race: If 2 concurrent requests arrive at the same millisecond, both pass a read check if key isn't stored yet. Solution: Use atomic SET key lock_token NX EX 86400 before starting processing.
  7. Store JSON response body and HTTP status code in Redis mapped to the Idempotency Key. When duplicate request arrives, return the stored status code and JSON body directly.
  8. IETF Draft Spec: Client sends Idempotency-Key: <UUID>. Server returns 400 Bad Request if key format is invalid, 409 Conflict if concurrent request in progress, and identical response payload on retries.
  9. TTLs (e.g. 24-72 hours) ensure Redis memory is reclaimed after retry windows pass. Most client network retries occur within seconds/hours of original failure.
  10. Pass the original Idempotency-Key downstream to third-party APIs (e.g. Stripe Idempotency-Key header), ensuring third-party downstream calls are also deduplicated.

8. Common Mistakes

  • Assuming POST endpoints are idempotent without explicitly implementing an Idempotency-Key framework.
  • Checking Redis for key existence and setting it in 2 non-atomic steps, allowing concurrent race condition bypasses.
  • Executing database updates before checking or locking the Idempotency Key in Redis/DB.
  • Returning an error payload on retries instead of returning the original successful cached response payload.
  • Generating Idempotency Keys on the server side instead of requiring the client application to generate UUIDs.

9. Keywords

Idempotency, Idempotency-Key, \(f(f(x)) = f(x)\), HTTP Verbs, SET NX, UNIQUE Constraint, Deduplication, Stripe API, Redis, Race Condition, 409 Conflict, Retries, IETF Draft, Atomic Check-and-Insert.

10. Cheat Sheet

  • Core Definition: Executing an operation 1 time or 100 times produces identical system state & response.
  • REST Verbs: GET, PUT, DELETE = Idempotent; POST, PATCH = NOT Idempotent (requires explicit keys).
  • Client Rule: Client MUST generate the UUID Idempotency-Key header and attach it to network retries.
  • Server Execution Flow:
  • Atomic SET idempotency:key IN_PROGRESS NX EX 86400 in Redis.
  • If Key Exists & IN_PROGRESS \(\rightarrow\) Return 409 Conflict.
  • If Key Exists & COMPLETED \(\rightarrow\) Return Cached JSON Response.
  • Process Business Logic in DB.
  • Update Redis key status=COMPLETED + Store JSON Response.
  • Database Safety: Add UNIQUE INDEX on (tenant_id, idempotency_key) in database tables as secondary safety net.

11. Related Decisions