How to save tasks that cannot run immediately
How tasks differ from immediate responses
Some work can finish after responding to the user: sending an email, processing a photo, preparing a report. The application creates a task describing the work, and a separate handler — a worker — executes it.
Workers have limited speed. Tasks arriving faster must be kept until a worker is free or rejected. Initially there is no waiting space, so excess tasks are lost immediately.
How a queue survives a burst
A queue buffers between task producers and workers. A free worker takes the next task; others wait. Queue capacity determines how many waiting tasks can be retained.
For example, 500 tasks arrive each second and workers process 200. The queue grows by 300 per second, reaching 1,200 after four seconds. When input drops to 100, workers drain backlog at 100 tasks/s: half their capacity serves new tasks.
Why a large buffer does not solve everything
Queues buy time but do not speed processing. If tasks continually arrive faster than they are completed, any finite buffer eventually fills. Here, new tasks beyond capacity are rejected.
Even without losses, users can wait too long. More workers speed processing and reduce backlog but cost money. Choose both processing rate and capacity within budget.
How to read experiment results
Open the analysis and compare the start of the burst, its end and the end of the experiment. Check accounting: every incoming task is completed, lost or still queued. Accepted does not mean completed.
Wait is estimated as peak backlog divided by total worker rate, not the exact wait of each task. This lesson omits worker failures and redelivery: first understand traffic, accumulation and processing speed.
Task
For the first 10 seconds the API creates 600 tasks/s, then drops to 100. One worker handles 100 tasks/s. Retain tasks during the peak and finish by second 60.
The queue starts empty. We model 60 one-second steps: 10 seconds of burst, then 50 seconds at 100 tasks/s. Tasks are identical; every worker reliably handles 100 tasks/s. This is a finite teaching flow, not a broker measurement. Animation shows routes; the chart shows all 60 seconds. Tasks arriving in a step can be processed in that same step; no unfinished in-flight tasks carry between steps. There are no retries, duplicates, failures, TTL, priorities or delivery guarantees. A larger queue does not increase processing speed and may increase waiting.
- Find the interval where more tasks arrive than finish.
- Add a queue and workers from the panel; compare accumulation and drain rates.
- Choose buffer size and capacity to finish on time without unnecessary loss or expense.