← Study Notes Backend


Backend

Queues

A buffer (RabbitMQ, SQS, Kafka) that decouples producers from consumers so slow or spiky work — emails, image processing, webhooks — runs asynchronously and survives restarts. It smooths load spikes and adds resilience. In exchange you accept eventual consistency, possible duplicate or out-of-order delivery, and the need for idempotent consumers and dead-letter queues.


Purpose

A message queue is a durable buffer between a producer and a consumer, so work can be handed off and processed asynchronously instead of blocking the request. It exists to decouple components, absorb spikes and make slow or failure-prone work reliable.

When to Use It

Sending email, processing images and video, delivering webhooks, order pipelines, and fanning work out to many workers — anything that need not finish inside the user's request. It also smooths bursty load into steady throughput.

For request/response you still want a direct call; queues are for fire-and-forget or background work.

Trade-offs

You gain resilience and decoupling but accept eventual consistency, harder debugging, and delivery quirks: messages can arrive out of order or more than once. Consumers must therefore be idempotent, and you need dead-letter queues for poison messages.

Implementation

Pick a broker — RabbitMQ or SQS for straightforward queues, Kafka when you need an ordered, replayable log. Make handlers idempotent, acknowledge only after success, retry with backoff, and route repeatedly-failing messages to a dead-letter queue for inspection.