The wait predicate was `stopped_ || total_ > 0`, and total_ counts queued
*plus executing*. So while any one task ran, every other worker's predicate
was true: wait() returned immediately and the worker spun through try_pop,
try_steal and back to wait at full speed, try_lock-ing every peer queue on
each pass.
Measured on this tree, 8 workers and one 300 ms task: 1991 ms of CPU and
19205 voluntary context switches, against 0.4 ms and 10 with the fix. Call it
six and a half cores burned for the duration of one sleeping task.
Sleeping requires "no work is *waiting*", which total_ cannot express, so
queued_ is now tracked separately: incremented on submit, decremented when a
task leaves a queue, and adjusted for the tasks stop() discards. total_ stays
as it was for drain(), which genuinely does need to know about executing work.
The worker exit condition moves to queued_ for the same reason — waiting for
total_ to reach zero meant waiting for someone else's task to finish, which a
worker cannot help with and would spin through until it did. PoolSnapshot's
queue depth stops being an estimate as a side effect.
Latent for this pipeline, where each node owns a private single-thread pool
and there is no idle peer to spin. Any use of a shared pool, which
make_pool_node exists for, hits it immediately.
Reproducing it needs the right trigger, and the test says so, because my first
attempt got it wrong and passed against the bug: a worker that has never been
woken stays blocked in wait() and never re-evaluates the predicate. The spin
only appears once a worker *finishes* something and re-enters the loop while a
peer is still busy, so the case submits one long task plus a trivial one per
remaining worker. Submitting only the long task measures nothing.
Verified in both directions: 1969 ms of CPU before, 0.35 ms after, against a
300 ms threshold. Full suite 142/142.
dtourolle
b9698fae60