Size the pool to the work: CPU-bound tasks want roughly N or N+1 threads (where N = available cores), because more than that just adds context-switch overhead with no parallelism gain. I/O-bound tasks want many more than N, because each thread spends most of its time blocked, leaving the CPU free.
CPU-bound
Compute-heavy work (hashing, parsing, image transforms) keeps a core busy the whole time. The optimal pool size is the number of cores; +1 keeps the CPU fed if a thread occasionally faults on a page.
int cores = Runtime.getRuntime().availableProcessors();
ExecutorService cpuPool = new ThreadPoolExecutor(
cores, cores, 0L, TimeUnit.MILLISECONDS,
new ArrayBlockingQueue<>(1000));I/O-bound — Brian Goetz's formula
When threads block on network or disk, the CPU sits idle and you want enough threads to keep it busy. From Java Concurrency in Practice:
threads = N_cpu * U_cpu * (1 + W/C)
where N_cpu = cores, U_cpu = target CPU utilization (0–1), and W/C = the ratio of wait time to compute time. A task that waits 90 ms on a DB and computes 10 ms has W/C = 9, so on 8 cores at full utilization you want 8 * 1 * (1 + 9) = 80 threads.
This is just Little's law in disguise: to sustain throughput λ with average latency L, you need λ * L concurrent workers in flight.
Practical caveats
- Separate pools per workload. Don't share one pool between fast CPU tasks and slow blocking calls — the blocking ones will hog every thread and starve the rest (the bulkhead pattern).
- Memory and downstream limits cap you. 80 threads is pointless if your DB connection pool is 20; size to the narrowest bottleneck.
- Virtual threads change the math. For purely blocking I/O on Java 21+, a virtual-thread-per-task executor sidesteps sizing entirely — you no longer pool to ration scarce platform threads.