Choosing a BullMQ concurrency limit
Set BullMQ concurrency from the scarce downstream resource and validate it under representative load. Account for every worker replica, distinguish in-flight work from requests per second, and stop increasing concurrency when latency or failure rises without useful throughput gains.
Inside this article 4 sections +
The right BullMQ concurrency setting is the highest useful level that stays inside your dependency budgets under realistic conditions. It is not the largest number that drains a synthetic queue quickly. A worker can look productive while moving its backlog into the database connection pool or an external provider's rate limiter.
Use an illustrative invoice-export workload: each job reads account data, renders a document, uploads it, and records a result. Those stages consume different resources. Increasing concurrency may overlap upload waits efficiently while creating too many simultaneous database reads or competing document renders.
Count capacity across the whole deployment
BullMQ's local concurrency controls a worker instance. If four replicas each allow eight concurrent jobs, the deployment can have up to thirty-two active jobs through those workers. An autoscaler changes that total even when nobody edits the concurrency setting. Record minimum, typical, and maximum replica counts beside the setting in the operating configuration.
Start with the resource that has the least spare capacity. Suppose a database connection budget reserves twelve connections for exports after interactive traffic and other jobs have their allocations. If an export can hold one connection throughout its execution, twelve simultaneous exports is already an upper boundary for that assumed design, before adding a safety margin. If jobs acquire connections only briefly, measure stage overlap rather than treating the boundary as a throughput prediction.
4 replicas × 8 jobs = 32 possible active jobs
- Database budget: 12 connections
- Determine how many active jobs actually hold a connection at the same time.
- Provider budget: requests per second
- Enforce separately; a concurrency limit does not establish a request-rate ceiling.
- Render budget: CPU and memory
- Measure contention and per-job memory before increasing parallel execution.
Measure where jobs spend their time
Instrument queue wait, total execution time, connection acquisition, document rendering, upload, and final persistence separately. Record completed business operations, attempts, failures, and retries. Counting attempts as throughput rewards a worker for repeating unsuccessful work. The useful result is completed exports at an acceptable age, not simply jobs started per second.
Watch the oldest eligible waiting job, not only queue length. A small queue can contain a large document that repeatedly loses access to a scarce resource. Segment by payload size or work class where that distinction changes cost. Avoid one metric label per job or customer; use traces and sampled diagnostic records for high-cardinality identifiers.
Node.js asynchronous I/O can benefit from overlapping work, but raising local concurrency does not make synchronous CPU computation run across more cores. A CPU-heavy render may require isolated processors or separate workers. Check event-loop delay and stalled-job events while testing; a queue that repeatedly loses worker liveness is not successfully tuned even if average throughput increases.
Run a bounded staircase experiment
Prepare a fixed dataset with small, typical, and unusually large exports. Include warm-cache and cold-cache behavior deliberately. Keep provider responses controlled in a test environment and record the database configuration. Start from a conservative setting, run long enough to observe steady behavior, then change only concurrency. Repeat the same workload at each step.
At every step, compare useful completion rate, queue age, execution percentiles, database wait, CPU, memory, and provider errors. Stop when the next increase delivers little useful throughput while wait or failure rises. Also stop at a resource budget, even if throughput is still increasing. This experiment identifies a range to investigate; it does not prove safety for every future workload.
Then run a disturbance test at the candidate setting. Slow uploads, introduce a database pause, and let one replica restart. Check whether outstanding work remains bounded and whether recovery creates an excessive burst. Reserve enough capacity for these conditions rather than configuring every dependency to operate at its observed peak all the time.
Make the limit survive autoscaling and retries
Choose a deployment-wide policy alongside the local setting. That may involve a shared admission mechanism, a queue-level global limit supported by the installed version, or a conservative replica cap. The mechanism matters less than the guarantee: adding replicas must not silently exceed the resource budget. Document which other queues share that budget.
Rate limits need a separate policy because rapid jobs can issue many requests despite low concurrency. Retries also consume provider capacity and should count against the same budget as first attempts. During an outage, pause or reduce affected work instead of allowing more replicas to amplify failed calls. Keep interactive and batch capacity isolated when their service objectives differ.
Roll out the candidate setting gradually and retain the previous configuration. Define reversal conditions before deployment, including sustained connection waits, rising throttling, or worsening queue age. The reliable jobs guide covers duplicate-safe recovery, and Kafka versus job queues clarifies the workload model. Effective background processing implementation connects those guarantees with measured capacity.