fix: a push must wake its consumer, even when the ring looked non-empty

push(), try_push() and push_blocking() fired push_callback_ only on the
empty->non-empty edge, and computed that edge from a head_ sampled before
the item was published. A PoolNode consumer decides whether to run again
from the level (count_ready -> approx_size), so a pop landing in that
window left both sides standing down:

  producer (push)                    consumer (PoolNode firing)
  ------------------------           ----------------------------
  samples t=782, h=781
    -> was_empty = false, no wake
                                     pops idx 781, head_ = 782
                                     count_ready(): head_==tail_==782
                                     -> not ready, gate released to Idle
  tail_.store(783)

The item is in the ring, the node is idle, and no wake is outstanding.
The failure is absorbing: every later push then sees a non-empty ring, so
the edge never fires again and the node sleeps while its backlog grows.

Observed as a hang in bench_pipeline at (chain, depth=4, work_us=10,
shared pool): all pool workers asleep in worker_loop, the reader blocked
in pop(), and 218 items stranded in one channel with head_ stopped at
exactly the index where the edge was dropped.

Re-reading head_ after the tail_ store does not fix this. That is the
store-buffer pattern, and under acquire/release both sides may legally
read stale; forbidding it needs seq_cst on the producer's tail_ store and
head_ load *and* on the consumer's head_ store and tail_ load, a fence on
both hot paths. Firing unconditionally is correct by construction: the
callback runs after the publishing store, so a consumer that observes the
level at all observes the item. The redundant wakes are cheap --
on_input_ready re-checks the level and SubmitGate::claim() collapses a
wake arriving mid-firing into the firing already in flight.

The stress test named this exact hazard and could not detect it: it
asserted only 1 <= callbacks <= N, which a *missed* callback satisfies.
It now requires one callback per successful push, and fails at 1325/10000
against the old code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

tests/test_channel_stress.cpp

@@ -153,11 +153,18 @@ TEST_CASE("SPSC: producer racing a disable() never throws and never hangs",
     }
 }
 
-TEST_CASE("SPSC: push_callback fires on each empty->non-empty transition",
+TEST_CASE("SPSC: push_callback fires for every push, never missed",
           "[channel][stress]") {
-    // The empty->non-empty callback ([channel.hpp] was_empty branch) is read by
-    // the consumer-side notification path. Run it under contention to make sure
-    // the was_empty detection isn't torn by a concurrent pop().
+    // Regression: this callback is the *only* thing that wakes a PoolNode, and
+    // it used to fire only on the empty->non-empty edge, computed from a head_
+    // sampled before the item was published. A concurrent pop() could drain the
+    // ring to empty in that window, so neither side saw the other: the item sat
+    // in the ring with the consumer idle, and because the trigger was an edge it
+    // never recovered. See set_push_callback in channel.hpp.
+    //
+    // The old version of this test asserted only `1 <= callbacks <= N`, which a
+    // *missed* callback satisfies — it named the hazard and could not detect it.
+    // One callback per successful push is the contract, so assert exactly that.
     Channel<int> ch(/*capacity=*/4, /*spin_count=*/4);
     std::atomic<int> callbacks{0};
     ch.set_push_callback([&] { callbacks.fetch_add(1, std::memory_order_relaxed); });
@@ -175,10 +182,9 @@ TEST_CASE("SPSC: push_callback fires on each empty->non-empty transition",
     for (int i = 0; i < N; ++i) (void)ch.pop();
     producer.join();
 
-    // At least one transition, at most one per item; mainly we assert the run
-    // completed without TSan flagging a race on push_callback_/was_empty.
-    REQUIRE(callbacks.load() >= 1);
-    REQUIRE(callbacks.load() <= N);
+    // Exactly one callback per successful push. Fewer means a wake was dropped,
+    // which is the bug; more would mean a spurious wake was manufactured.
+    REQUIRE(callbacks.load() == N);
 }
 
 // Ordering contract of the out-of-band sentinel under contention.