Libuv threadpool sizing: results and tradeoffs
Where Nub landed
Nub retained CPU-based automatic threadpool sizing; it did not add a memory-based cap or revert every process to four workers. This repository records the measurements behind that decision and the limitations of the policy.
For the evaluated policy in Nub 0.9.3:
- Automatic size starts at
max(4, available_parallelism). Detected process/thread headroom can reduce it, with a floor of four; Windows additionally caps it at eight.
- On Linux, workers beyond the first four run at nice 10 on a best-effort basis.
- An explicit
UV_THREADPOOL_SIZE wins. Compatibility mode leaves Node's default alone. Nub's automatically chosen value is not inherited by plain Node child processes; a child launched through Nub is sized again.
- There is no automatic memory-safety guarantee. CPU availability, nice values and cgroup weights do not reveal how much memory future native operations will allocate.
The recorded decision is nubjs/nub#947. The sizing implementation and priority/child policy are pinned to the source inspected for this study.
The conclusion is a tradeoff, not a universal optimum. Larger pools produced substantial gains in several tested workloads, but also caused OOMs and reduced a neighbouring cgroup's throughput. Four avoided some failures and still failed others. The experiments did not establish a better general-purpose replacement default.
Main results
The two Linux campaigns contain 248 trials, plus four identity records. All observations below have raw records and executable fixtures in this repository; the older HTTP result is included separately with its original provenance.
| Question |
Observation |
Limit of the conclusion |
| Can a larger pool help? |
At 1 GiB, a 32-job scrypt batch took 721 ms with four Nub workers and 327 ms with automatic sixteen. |
One workload and allocation, not a universal speedup. |
| Can it cause OOM despite nice 10? |
At 384 MiB, the same 32-MiB-per-operation workload completed with four and was OOM-killed with automatic sixteen, 3/3 trials. |
A finite burst, not an unbounded producer. |
| Does niceness protect a neighbouring cgroup? |
On a 16-vCPU host, automatic sixteen increased Nub's PBKDF2 throughput by 69% but reduced an equal-weight sibling's throughput by 21% versus four. Extra workers were verified at nice 10. |
Cgroup isolation still worked. More parallel work consumed capacity the neighbour had previously borrowed. |
| Does a generic memory cap solve it? |
A prototype choosing one worker per 64 MiB fixed the 32-MiB-operation case, then failed with 64-MiB operations where four workers completed. |
The tested constant is not a production-ready memory model. This does not disprove every possible heuristic. |
| Is four a memory-safe fallback? |
At 384 MiB, four workers were also OOM-killed with 128-MiB scrypt operations, 3/3 trials. |
Pool size alone cannot bound arbitrary operation memory. |
| Does the linked gzip example require a large pool to OOM? |
The unbounded producer was OOM-killed with both one and four workers. Its bounded-submission controls survived the watchdog window. |
That example demonstrates queued-input pressure, not an autosizing comparison. |
Times are medians of three trials unless stated otherwise. See results for budgets, parameters, memory peaks, historical tail latency, and the positive and negative controls.
What the admission-control experiment does—and does not—show
The experiment was given the memory cost; Nub did not discover it. The scrypt fixture supplies an approximate 128 × N × r working-set estimate, its deliberately retained buffer size, and a chosen half-memory reserve.
Keeping sixteen workers while admitting eight known 64-MiB operations at a time completed a 1-GiB batch in 841 ms, versus 1477 ms with four workers. But this is a controlled application-level example, not a generic limiter for arbitrary Node code. The reserve is a heuristic, and the estimate excludes allocator retention, other jobs and future allocations. Even the fixed admission-four control had much higher peaks than a four-worker pool in one case.
For general image processing, native addons, mixed work and input-dependent allocations, the memory requirement may not be knowable before execution. Neither a startup launcher nor the successful controlled example establishes otherwise. Measurement can guide deployment-specific limits; it cannot turn them into a guarantee for unknown inputs.
Read or reproduce
The saved data can be validated without running workloads or inducing OOM:
python3 scripts/verify.py
Live probes require a disposable Linux VM with cgroup v2 and administrative access. They deliberately induce OOM inside constrained groups. Do not run them on a production or shared machine.
Motivation and references
The investigation started from the risks discussed in platformatic/gzip-eval-async. Those risks are real; the purpose here is to distinguish queue growth, executing native work, within-group priority and cross-group CPU sharing.