3 Commits

Include renames

Author	SHA1	Message	Date
dtourolle	a8cfe7300a	fix: a lossless fanout, a node that starts awake, and the instrumentation that found them ... Three changes from one debugging session on the intermittent wedge, kept together because the instrumentation is what made the other two findable. **Fanout was never made lossless.** 6595e6e made node outputs lossless and 28e0667 stopped them parking a worker; FanoutNode was in neither and kept `catch (ChannelOverflowError&) {}` per output. Whichever branch fell behind lost items, silently, by an amount that depended on timing — so two runs of the same input could disagree. deliver() now retries each output independently until it is taken, rechecking stop_flag_ every pass so teardown cannot hang on a full output. A fanout owns a private thread, so waiting costs no scheduler worker. **A node could start with a wake already outstanding.** start() enables the input channel several statements before it installs the push callback, and StaticNetwork starts nodes sources-first, so an upstream node is already firing into the gap. A push landing there is accepted by the ring but wakes nobody: push_callback_ fires only on the empty→non-empty transition, and at that instant the callback is null. Every later push sees a non-empty ring and stays silent, so the node is never submitted. Asking on_input_ready() once at the end of start() converts the missed edge into a state check. The signature is distinctive — zero items delivered, not a stall partway. Under `ctest -j4` on a loaded machine it reproduced 7 times in 24 and never in 10 unloaded runs, which is almost certainly the "~1 run in 20" hang 28e0667 recorded as known-incomplete. **NodeSnapshot now carries scheduling state and a true exec total.** queued and wake_pending make the 9c5ce5f invariant observable at runtime; it could previously only be inspected in a debugger, and the bug does not reproduce under one. total_exec_us is a real sum — frames × ema_exec_us tracks the tail of a run, not the whole of it, and diverges badly on a workload whose per-frame cost varies. Both are exposed over the web debug JSON so a wedged pipeline can be interrogated without attaching to it.	2026-08-05 12:40:03 +02:00
dtourolle	9c5ce5f34a	fix: never drop a wake — a node must not sleep with one outstanding ... 28e0667 stopped nodes blocking a worker on a full output, but replaced an intermittent hang with a quieter one: the pipeline still wedged about 2 runs in 30, now with every worker idle in pthread_cond_wait rather than asleep in a push. Nothing was blocked; nothing had been woken. try_submit discarded any wake arriving while queued_ was up: if (queued_.compare_exchange_strong(expected, true, ...)) scheduler_->submit(...); // else: silently gone Wakes are edge-triggered — a channel fires its space callback once, on the transition — so a dropped one never returns. A node could park a value, release its worker, and sleep forever holding exactly the output its consumer was waiting for, while its producer parked on an input channel that would never drain. try_submit now records the drop in wake_pending_, and release_and_recheck() consumes it at every site that releases a node, giving one invariant: a node never sleeps with a wake outstanding. This subsumes the two ad-hoc re-checks added for the parked-retry and normal push paths, which only moved the stall (1211 items to 6472) because each new early return was a fresh chance to drop a wake. self_stop keeps a plain store — honouring a pending wake there would resubmit a dead node. Adds "a saturated chain never stalls", which reproduces this in about a second where the pipeline needed ~30 runs. It asserts *progress does not freeze* rather than a completion total: capacity-1 channels are slow, and slow must never be reported as wedged. Verified in both directions — it stalls after 1211 items on 28e0667 and passes here. The existing chain test could not catch it: 40 items drain before any strand occurs, and its producer emits forever, so fresh input keeps re-triggering on_input_ready() and flushing the stranded value. Tests: 122/122.	2026-08-02 18:33:17 +02:00
dtourolle	28e06675f5	fix: park nodes on a full output instead of blocking the worker ... push_blocking parked a scheduler worker inside the push. Nodes own a private single-thread pool, so the parked thread was the only one that could drain that node's own input — hold-and-wait, and under sustained backpressure four nodes of a five-node chain slept in nanosleep at once. channel.hpp already warned about this for sentinels; it applies just as much to data pushes. The scheduler was purely input-driven: on_input_ready() wakes a node when input arrives, with no counterpart for "my output has room". Lacking that signal, blocking the thread was the only way to handle a full output. This adds the missing half. - Channel::try_push + has_space + set_space_callback; the callback fires from both pop() and try_pop_now(). - PoolNode/PoolObjectNode keep a one-slot pending_ buffer with per-element done flags, so a retry cannot duplicate an already-accepted element. One slot suffices because queued_ admits at most one fire_once per node. - The re-check after clearing queued_ closes the lost-wakeup race where a space callback fires while the flag is still up and is swallowed. Two bugs surfaced once nodes actually parked, both fixed here: - pop_one reports an *empty* channel as ChannelClosedError, which is also the node's "upstream finished, self-stop" signal. A node woken by output space with empty inputs therefore killed itself. fire_once now releases the worker when its inputs are not ready rather than falling through. - The drained-park path resubmitted unconditionally instead of via on_input_ready(), firing nodes with nothing to read. compute_priority is now output-aware: mean output fill is deducted from mean input fill, mapped as 0.5·(1 + in - out). Input fill alone asks only "how much work is waiting for me"; a node whose outputs are already full cannot deliver, so running it just parks it again and wastes the slot while the node that would drain that channel waits behind it. The scheduler now favours whoever is furthest downstream of a bottleneck. Also adds a network-level error listener. A node's exception was discarded at the node boundary and survived only as a Closed event, which reports that a node stopped but not why — that missing detail is what made the above slow to diagnose. INode::set_network_error_callback plus StaticNetwork::set_error_handler forward it to the application. Tests: 121/121. test_backpressure_deadlock drives a five-node chain with capacity-2 channels against a slow sink and fails on the old code. The four test_pool_node overflow tests now assert parking rather than the removed drop-and-report behaviour. Known-incomplete: a rare hang remains, roughly 1 run in 20 against a 300s timeout, down from every run failing. Committed because the fix is a large strict improvement and the residual case needs its own reproduction.	2026-07-31 22:40:07 +02:00