CSharpMind

Where Thread Pool Starvation Actually Comes From

Blocking on a task is the case the thread pool compensates for. Blocking on a sleep or an event is the case it cannot see, and it finished 2.5 times slower.

5 min read
A pinned programme barrel lying on a workbench, its surface covered in identical upright pins, with a bank of brass levers riding the row nearest the camera and one violet lever among them. A blank paper tag hangs from a wire.

Two hundred work items, each holding a pool thread for half a second, on a twelve-core machine. Blocking on a task, the run finished in 3.2 seconds. Changing the blocking call to Thread.Sleep and nothing else, the same run took 8.0 seconds.

The work is identical. What differs is whether the thread pool was told that its threads were blocked.

What the pool did in each case

The measurement queues 200 items through Task.Run, samples ThreadPool.ThreadCount and ThreadPool.PendingWorkItemCount from a dedicated foreground thread — a sampler queued to the pool would be starved by the very condition it is measuring — and varies only the call that occupies the worker.

Blocking on a task, with Task.Delay(500).Wait():

 t (ms)  threads  queued   done
    252       33     167      0
    753       44     151     19
   1509       55     114     52
   2264       63      64    100
   2767       68       0    163
wall clock 3182 ms, final ThreadCount 68

Blocking on a sleep, with Thread.Sleep(500):

 t (ms)  threads  queued   done
    252       12     188      0
   1008       12     164     24
   2519       12     128     60
   4027       13      91     96
   6543       13      26    161
wall clock 8026 ms, final ThreadCount 13

ManualResetEventSlim.Wait(500) behaves like the sleep: 12 threads, rising to 15 over seven seconds, 7524 ms in total.

The first run injected 56 threads in under three seconds. The second injected one. Both were blocked in exactly the same sense — a worker occupied, doing nothing, while a queue built up behind it.

The mechanism, and where it is written down

The difference is a notification. Since .NET 6, Task.Wait tells the thread pool that the thread it is running on is about to block, and the pool responds by injecting threads far more aggressively than its normal hill-climbing heuristic would. The runtime configuration knobs for that behaviour are documented, and the documentation is unusually direct about the limit:

Currently, these settings take effect only for work items that wait for another task to complete, such as in typical sync-over-async cases.

The knobs say how the injection is paced. After the thread count based on MinThreads is reached, ThreadsToAddWithoutDelay more threads may be created with no delay at all; after that a delay of DelayStepMs is induced before each new thread, growing by another DelayStepMs for every ThreadsPerDelayStep threads, capped at MaxDelayMs. In the runtime’s own thread pool the two thread counts default to one times the processor count, the delay step to 25 ms and the cap to 250 ms.

Those defaults predict the first sample. Twelve cores gives twelve minimum threads and twelve more with no delay, so the pool should reach 24 immediately and then add roughly one thread per 25 ms. The sample at 252 ms shows 33 threads — 24 free, then nine more at a step of 25 ms. The mechanism is not merely documented; it is arithmetically visible in the trace.

Nothing in that path is reachable from a Thread.Sleep, a Monitor wait, an event, a semaphore or a blocking file read. NotifyThreadBlocked in the runtime returns immediately unless the calling thread is a pool thread and cooperative blocking is enabled, and nothing calls it on behalf of those primitives. The pool does not know the threads are gone; it sees a work item that has not returned yet, which from the outside looks the same as a work item that is busy.

There is an open API proposal to let application code make the same notification for blocking the runtime cannot detect. Until something like it exists, the asymmetry is a property of the platform rather than a configuration mistake.

Why this reverses the usual advice

The standard guidance is to eliminate sync-over-async, and it is good guidance — a blocked worker is still a worker not doing work, and in a context with a single-threaded synchronisation context the pattern deadlocks outright rather than merely costing throughput. But it is aimed at the one form of blocking the runtime already compensates for. That has two practical consequences.

An incident that survives the removal of every .Result and .Wait() was probably not caused by them. The remaining suspects are the blocking the pool cannot see: a lock held across a slow operation, a SemaphoreSlim.Wait on a saturated connection pool, a synchronous HTTP or database call inside a handler, a Task.Run wrapping a synchronous library. Those are the calls that hold the thread count flat while the queue grows.

And the shape of the incident tells you which one you have before you read any code. Two counters are enough. If ThreadPool.ThreadCount is climbing steadily while PendingWorkItemCount grows, the pool has detected the blocking and is compensating — slowly, because injection is deliberately paced, but it is compensating. If the thread count is sitting near the processor count while the queue grows without bound, nothing told the pool anything, and no amount of waiting will fix it.

Raising MinThreads works in both cases, and it is the right emergency lever. It is also a fixed number picked in advance for a load that is not fixed, and it leaves the blocking call in place. The durable fix is to make the call asynchronous, or to move it off the pool entirely onto a dedicated thread, which costs one thread rather than an unbounded share of the pool’s.

That second option is worth stating precisely, because it is usually reached for through the wrong door. Task.Run puts the work back on the pool, which is the resource being protected; a long-lived blocking loop belongs on a Thread of its own, or behind a bounded consumer reading from a channel. The rule of thumb that survives both cases is narrow: a work item queued to the pool should be short, and it should end by returning rather than by waiting.

None of this is visible from the source of an individual method, which is the difficulty: the same await that costs nothing when it completes synchronously — as the state machine article measured — becomes a pool-wide event the moment something blocks in front of it.


Measured on .NET 10.0.11, macOS 26.6.2 arm64, 12 cores, Release configuration. ThreadPool.GetMinThreads reported 12 worker threads at start. Each run is a fresh process; the sampler is a dedicated foreground thread rather than a queued work item. The documented quotation and the injection settings are from the .NET threading configuration reference.

Frequently asked

Does this mean sync-over-async is safe?
No. It still burns a pool thread per outstanding operation and it still deadlocks in contexts with a single-threaded synchronisation context. The finding is narrower — sync-over-async is the blocking pattern the pool detects and compensates for, so an incident that survives its removal was probably never caused by it.
How do I tell which kind of blocking my process is doing?
Watch ThreadPool.ThreadCount against ThreadPool.PendingWorkItemCount. A queue that grows while the thread count climbs steadily is blocking the pool can see. A queue that grows while the thread count sits at roughly the processor count is blocking it cannot.
Can I raise the minimum thread count instead?
It works, and it is the standard mitigation, but it is a fixed number chosen in advance for a load that varies. It also does nothing about the underlying blocking. Treat it as the thing that buys time while the blocking call is made asynchronous.
Share

Related articles

Arrow keys to move, Enter to open.