Deliver EOF sentinels out-of-band to prevent teardown deadlock
Channel::push() drops values on overflow (the intended backpressure policy for data), and PoolNode swallows the resulting ChannelOverflowError. For a control sentinel like EOF this is fatal: a single dropped EOF under backpressure wedges every downstream pop() forever, so the pipeline never tears down. Deliver sentinels out-of-band instead. Channel::push_sentinel() stores the token in a dedicated slot that does not consume ring capacity, so it can never overflow and — crucially — never blocks the caller. That non-blocking property is essential: each KPN node has a single worker thread, so a *blocking* push would park that thread and stop it draining its own input, cascading into a hold-and-wait deadlock under backpressure. The consumer's pop()/try_pop_now() drain the ring first, then deliver the sentinel, so it always arrives after every value pushed before it. approx_size() (which node readiness checks call) counts a pending sentinel as consumable work, so a channel carrying only a sentinel still schedules its consumer's next fire — without this the token would sit undelivered and the pipeline would still deadlock at teardown. PoolNode/PoolObjectNode route values carrying an eof flag (direct .eof or nested .source.eof) through push_sentinel via a SFINAE-safe is_sentinel_value trait; all other values keep the existing lossy throwing push. The trait compiles to false for types without an eof convention, so this is a no-op for pipelines that don't use one. Verified end-to-end: scene_analyze now reaches EOF, flushes its output, and exits cleanly instead of hanging. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -22,6 +22,38 @@
|
||||
|
||||
namespace kpn {
|
||||
|
||||
// ── Sentinel detection ────────────────────────────────────────────────────────
|
||||
// A value is a "sentinel" (must-deliver control token, e.g. EOF) if its type
|
||||
// carries a bool-convertible eof flag — either directly (`v.eof`, as on a raw
|
||||
// source Frame) or nested one level under a `.source` member (`v.source.eof`,
|
||||
// as on the pipeline's SceneFrame/…/MatchedSceneFrame message types, which wrap
|
||||
// the originating Frame). Sentinels are delivered losslessly and non-blockingly
|
||||
// via Channel::push_sentinel() instead of the throwing push(), so backpressure
|
||||
// can never drop the token that unblocks downstream teardown.
|
||||
//
|
||||
// Types with neither shape are never treated as sentinels — both traits are
|
||||
// SFINAE-safe and the runtime check compiles away to `false` for them, so this
|
||||
// stays a no-op for pipelines that don't use an eof convention.
|
||||
template<typename T, typename = void>
|
||||
struct has_eof_field : std::false_type {};
|
||||
template<typename T>
|
||||
struct has_eof_field<T, std::void_t<decltype(static_cast<bool>(std::declval<const T&>().eof))>>
|
||||
: std::true_type {};
|
||||
|
||||
template<typename T, typename = void>
|
||||
struct has_source_eof_field : std::false_type {};
|
||||
template<typename T>
|
||||
struct has_source_eof_field<T,
|
||||
std::void_t<decltype(static_cast<bool>(std::declval<const T&>().source.eof))>>
|
||||
: std::true_type {};
|
||||
|
||||
template<typename T>
|
||||
constexpr bool is_sentinel_value(const T& v) {
|
||||
if constexpr (has_eof_field<T>::value) return static_cast<bool>(v.eof);
|
||||
else if constexpr (has_source_eof_field<T>::value) return static_cast<bool>(v.source.eof);
|
||||
else return false;
|
||||
}
|
||||
|
||||
// ── PoolNode ──────────────────────────────────────────────────────────────────
|
||||
//
|
||||
// Reactive alternative to Node<>. Instead of owning a blocked thread, the node
|
||||
@@ -361,6 +393,13 @@ private:
|
||||
void push_one_out(std::tuple_element_t<I, return_tuple>&& val) {
|
||||
auto* ch = std::get<I>(output_channels_);
|
||||
if (!ch) return;
|
||||
// Sentinels (EOF) must never be dropped: a lost token wedges every
|
||||
// downstream pop() forever. Deliver them out-of-band (push_sentinel),
|
||||
// which never overflows and never blocks this node's worker thread.
|
||||
if (is_sentinel_value(val)) {
|
||||
ch->push_sentinel(std::move(val));
|
||||
return;
|
||||
}
|
||||
try {
|
||||
ch->push(std::move(val));
|
||||
} catch (const ChannelOverflowError&) {
|
||||
@@ -660,6 +699,13 @@ private:
|
||||
void push_one_out(std::tuple_element_t<I, return_tuple>&& val) {
|
||||
auto* ch = std::get<I>(output_channels_);
|
||||
if (!ch) return;
|
||||
// Sentinels (EOF) must never be dropped: a lost token wedges every
|
||||
// downstream pop() forever. Deliver them out-of-band (push_sentinel),
|
||||
// which never overflows and never blocks this node's worker thread.
|
||||
if (is_sentinel_value(val)) {
|
||||
ch->push_sentinel(std::move(val));
|
||||
return;
|
||||
}
|
||||
try {
|
||||
ch->push(std::move(val));
|
||||
} catch (const ChannelOverflowError&) {
|
||||
|
||||
Reference in New Issue
Block a user