Skip to content

Pub/Sub vs Queue

1. Decision

Choose a Message Queue (Point-to-Point) when a single published task must be processed by exactly one competing worker node; choose Publish-Subscribe (Pub/Sub) when a single published event must be broadcast and processed independently by multiple subscriber microservices.

2. 30-Second Answer

Message Queues (Point-to-Point) implement 1:1 message distribution. Producers publish messages to a named queue; multiple worker instances pull from the queue as competing consumers, but each individual message is delivered to and processed by exactly one worker. Queues excel at load balancing background work tasks. Publish-Subscribe (Pub/Sub) implements 1:N event distribution. Publishers broadcast an event to a Topic without knowing who the receivers are. Multiple independent subscribers register interest in that Topic; every subscriber receives its own complete copy of the published event. Pub/Sub excels at event-driven microservice decoupling.

3. Decision Matrix

Option Best For Avoid When Latency Scalability Consistency Operational Complexity
Message Queue (Point-to-Point) Distributed task worker pools, asynchronous background processing, order processing jobs Broadcast event notifications to multiple independent microservice teams Low (<5ms task dispatch) High (Compacting workers scale horizontally) Transient; Message processed once and deleted Low
Publish-Subscribe (Pub/Sub) Event-Driven Architectures, broadcasting state changes (OrderPlaced), decoupling microservice domain events Single-worker background job execution where duplicate execution across services is prohibited Low (<10ms broadcast fanout) Extremely High (Subscribers scale independently per domain) Independent per subscriber (Eventual consistency) Medium

4. Selection Rules

Requirement Best Choice Rationale / Example
Competing Background Worker Job Execution Message Queue 10 email worker nodes pull tasks from queue; only 1 node sends the email
Broadcast Single Event to 5 Microservices Pub/Sub OrderPlaced event fans out to Billing, Inventory, Shipping, Analytics, & Email services
Add New Subscriber Without Modifying Publisher Pub/Sub New Fraud Detection service subscribes to existing PaymentCreated topic without code changes
Load Balance Heavy Compute Across Worker Nodes Message Queue Competing workers pull heavy image processing tasks off a shared queue
SNS + SQS Fanout Architecture Pattern Hybrid (Pub/Sub + Queue) SNS Topic (Pub/Sub) broadcasts message to 5 dedicated SQS Queues (Point-to-Point)

5. Tradeoffs

  1. 1:1 Task Distribution vs 1:N Event Broadcasting
  2. Gain (Queue): Work is divided evenly among workers; zero duplicate work execution.
  3. Lose (Queue): Cannot notify secondary systems without publishing duplicate messages.
  4. Decoupled Architecture (Pub/Sub) vs System Complexity
  5. Gain (Pub/Sub): Publishers have zero knowledge of downstream consumers; new features plug in seamlessly.
  6. Lose (Pub/Sub): Harder to reason about overall system execution flow and debug cascading event handlers.
  7. Competing Consumer Scalability vs Independent Domain Consumption
  8. Gain (Queue): Add more worker instances to speed up backed-up task queues.
  9. Lose (Queue): All worker instances must run identical code servicing the same queue logic.
  10. Message Lifecycle Economics
  11. Gain (Queue): Deletes message immediately after single worker ACK, saving memory/disk space.
  12. Lose (Pub/Sub): Broker must maintain copies or tracking pointers for N separate subscriber queues.
  13. Fanout Latency Overhead
  14. Gain (Pub/Sub): Asynchronous parallel delivery to N subscribers.
  15. Lose (Pub/Sub): Slow subscriber queues can cause broker memory backpressure if un-isolated.

6. Interview Counter Questions

  1. What is the fundamental difference between a Competing Consumer (Queue) and a Subscriber (Pub/Sub)?
  2. How do you implement a hybrid Pub/Sub + Queue architecture using AWS SNS and AWS SQS?
  3. How does Apache Kafka combine both Queueing and Pub/Sub semantics within Consumer Groups?
  4. How do RabbitMQ Fanout Exchanges implement Pub/Sub messaging?
  5. What happens in Pub/Sub if a new subscriber registers after an event was published?
  6. How do you handle message delivery failure in 1 subscriber without impacting other subscribers in Pub/Sub?
  7. What is Topic Filtering / Subscription Filters in Pub/Sub (AWS SNS / Google Cloud Pub/Sub)?
  8. Why is Pub/Sub essential for Event-Driven Microservice Architectures (EDAs)?
  9. How do you maintain transactional consistency across multiple Pub/Sub subscribers?
  10. When would you use Google Cloud Pub/Sub over Apache Kafka?

7. Gold Standard Answers

  1. Competing Consumer: Multiple workers listen to 1 queue, but only 1 worker processes each message. Subscriber: Multiple independent services listen to 1 topic, and every service receives its own copy of the message.
  2. Publisher sends message to AWS SNS Topic (Pub/Sub). SNS fans out the message to 5 separate AWS SQS Queues (Queuing). 5 distinct worker pools pull from their respective SQS queues independently.
  3. If all consumers share the same Consumer Group ID, Kafka acts as a Queue (competing consumers split partitions). If consumers use different Consumer Group IDs, Kafka acts as Pub/Sub (each group gets a full stream copy).
  4. Publisher sends message to a Fanout Exchange. The Fanout Exchange duplicates and routes the message to all bound queues. Competing worker pools read from their respective queues.
  5. In transient Pub/Sub (Google Cloud Pub/Sub, SNS), historical events published prior to subscription are missed. In log-based streaming Pub/Sub (Kafka, Pulsar), the new subscriber can rewind offsets to read historical events.
  6. Each subscriber domain has its own dedicated queue/subscription buffer. If Subscriber A fails or crashes, its dedicated queue buffers messages or dead-letters, while Subscribers B, C, and D process normally.
  7. Subscription Filters evaluate message attributes on the broker before delivery, allowing subscribers to receive only relevant subset messages (e.g., filter region = 'EU') without fetching all topic events.
  8. Pub/Sub enforces domain autonomy: domain microservices emit business facts (OrderPlaced) without knowing or caring which services react, eliminating tight coupling and breaking monolithic dependencies.
  9. Pub/Sub enforces Eventual Consistency. Each subscriber processes events in its own local transaction using Saga or Idempotent patterns; strict global distributed transactions (2PC) across subscribers are avoided.
  10. Choose Google Cloud Pub/Sub for fully managed, serverless, zero-ops global event broadcasting on GCP. Choose Kafka when log retention, event replayability, and custom stream processing (Flink) are required.

8. Common Mistakes

  • Using a Message Queue to send an event that 3 different microservices need to process (only 1 service gets it!).
  • Using Pub/Sub for a worker task pool, causing 10 identical worker nodes to execute the exact same email task 10 times.
  • Coupling Pub/Sub publishers to subscriber implementations by expecting synchronous return values.
  • Failing to isolate subscriber queues in Pub/Sub, allowing one slow subscriber to block message delivery to all others.
  • Expecting transient Pub/Sub systems (SNS) to store historical events for new subscribers without attaching persistent queues.

9. Keywords

Point-to-Point, Message Queue, Pub/Sub, Competing Consumers, Subscriber, Fanout, SNS + SQS Pattern, Consumer Group, Event-Driven Architecture (EDA), Subscription Filters, Eventual Consistency, Domain Autonomy.

10. Cheat Sheet

  • Core Difference: Message Queue = 1:1 distribution (Competing workers process task ONCE); Pub/Sub = 1:N distribution (Every subscriber gets a COPY).
  • Queue Use Case: Background job worker pools, task distribution, load balancing heavy compute.
  • Pub/Sub Use Case: Broadcasting domain events (OrderPlaced), decoupling microservices, event-driven architectures.
  • SNS + SQS Pattern: SNS (Pub/Sub) broadcasts event \(\rightarrow\) SQS Queues (Point-to-point) buffer tasks for independent worker pools.
  • Kafka Dual Model: Same Consumer Group = Queue; Different Consumer Groups = Pub/Sub.
  • Decoupling Rule: Publishers emit facts without knowing subscribers exist. Subscribers handle events in local transactions.

11. Related Decisions