Three changes from one debugging session on the intermittent wedge, kept
together because the instrumentation is what made the other two findable.
**Fanout was never made lossless.** 6595e6e made node outputs lossless and
28e0667 stopped them parking a worker; FanoutNode was in neither and kept
`catch (ChannelOverflowError&) {}` per output. Whichever branch fell behind
lost items, silently, by an amount that depended on timing — so two runs of
the same input could disagree. deliver() now retries each output
independently until it is taken, rechecking stop_flag_ every pass so
teardown cannot hang on a full output. A fanout owns a private thread, so
waiting costs no scheduler worker.
**A node could start with a wake already outstanding.** start() enables the
input channel several statements before it installs the push callback, and
StaticNetwork starts nodes sources-first, so an upstream node is already
firing into the gap. A push landing there is accepted by the ring but wakes
nobody: push_callback_ fires only on the empty→non-empty transition, and at
that instant the callback is null. Every later push sees a non-empty ring
and stays silent, so the node is never submitted. Asking on_input_ready()
once at the end of start() converts the missed edge into a state check.
The signature is distinctive — zero items delivered, not a stall partway.
Under `ctest -j4` on a loaded machine it reproduced 7 times in 24 and never
in 10 unloaded runs, which is almost certainly the "~1 run in 20" hang
28e0667 recorded as known-incomplete.
**NodeSnapshot now carries scheduling state and a true exec total.** queued
and wake_pending make the 9c5ce5f invariant observable at runtime; it could
previously only be inspected in a debugger, and the bug does not reproduce
under one. total_exec_us is a real sum — frames × ema_exec_us tracks the
tail of a run, not the whole of it, and diverges badly on a workload whose
per-frame cost varies. Both are exposed over the web debug JSON so a wedged
pipeline can be interrogated without attaching to it.