feat: opt-in lossless (blocking) output for nodes and fanouts
Channels drop on overflow by default. That is the right behaviour for live
sources, where a stale item is worth less than a fresh one, but it makes the
library unusable for offline batch work: items vanish with no diagnostic, and
any downstream analysis that assumes a fixed sample rate is silently invalid.
Auto-inserted fanouts were the harder half of this. They are created inside
make_network(), so user code cannot reach them to configure, and they drop
per-output inside a swallowed catch — so a pipeline whose own nodes were all
configured lossless could still lose items with nothing reported anywhere. In
the pipeline this came from, capture's 2505 frames arrived at the detector as
774 while the overflow counter read zero.
Adds:
- INode::set_lossless_output(bool), defaulted to a no-op so node types with
no output channels ignore it
- PoolNode / PoolObjectNode: route push_one_out() through push_blocking()
- FanoutNode: same, plus set_lossy_output(i) to opt a single branch back out
- StaticNetwork::set_lossless(), which reaches user nodes and fanouts alike
- StaticNetwork::drain(), a public wrapper over the existing private
drain_all_channels(), so callers can flush in-flight work before stop()
Default behaviour is unchanged; every path is off unless explicitly enabled.
Blocking output is only safe when every consumer eventually drains. A branch
that can stall indefinitely — a display node nobody is servicing — will apply
backpressure to the whole pipeline, which is what set_lossy_output() is for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -133,6 +133,14 @@ public:
|
||||
void set_error_handler(NodeErrorHandler h) { error_handler_ = std::move(h); }
|
||||
void set_max_exec_time(std::chrono::milliseconds t) { max_exec_time_ = t; }
|
||||
|
||||
// Lossless output: block the producer until the consumer drains rather than
|
||||
// dropping on a full channel. Default is drop, which suits live sources
|
||||
// where a stale frame is worth less than a fresh one. Enable for offline
|
||||
// batch runs, where a dropped item leaves a gap that downstream analysis
|
||||
// cannot recover. Must be set before start().
|
||||
void set_lossless_output(bool on) override { lossless_ = on; }
|
||||
void set_lossless(bool on) { set_lossless_output(on); }
|
||||
|
||||
void set_overflow_callback(NodeEventCallback cb) { event_callbacks_[0] = std::move(cb); }
|
||||
void set_network_overflow_callback(NodeEventCallback cb) override { event_callbacks_[1] = std::move(cb); }
|
||||
void set_closed_callback(NodeEventCallback cb) { closed_callbacks_[0] = std::move(cb); }
|
||||
@@ -400,6 +408,15 @@ private:
|
||||
ch->push_sentinel(std::move(val));
|
||||
return;
|
||||
}
|
||||
// Lossless mode: block until the consumer drains instead of dropping.
|
||||
// Dropping is the right default for live sources (a stale frame is
|
||||
// worth less than a fresh one), but for offline batch work every sample
|
||||
// matters — dropped frames leave a non-uniformly sampled series, which
|
||||
// silently invalidates any fixed-rate spectral analysis downstream.
|
||||
if (lossless_) {
|
||||
ch->push_blocking(std::move(val));
|
||||
return;
|
||||
}
|
||||
try {
|
||||
ch->push(std::move(val));
|
||||
} catch (const ChannelOverflowError&) {
|
||||
@@ -422,6 +439,7 @@ private:
|
||||
|
||||
std::shared_ptr<IScheduler> scheduler_;
|
||||
std::string name_;
|
||||
bool lossless_{false};
|
||||
std::size_t fifo_capacity_;
|
||||
input_channels_t input_channels_;
|
||||
output_channels_t output_channels_{};
|
||||
@@ -498,6 +516,14 @@ public:
|
||||
void set_error_handler(NodeErrorHandler h) { error_handler_ = std::move(h); }
|
||||
void set_max_exec_time(std::chrono::milliseconds t) { max_exec_time_ = t; }
|
||||
|
||||
// Lossless output: block the producer until the consumer drains rather than
|
||||
// dropping on a full channel. Default is drop, which suits live sources
|
||||
// where a stale frame is worth less than a fresh one. Enable for offline
|
||||
// batch runs, where a dropped item leaves a gap that downstream analysis
|
||||
// cannot recover. Must be set before start().
|
||||
void set_lossless_output(bool on) override { lossless_ = on; }
|
||||
void set_lossless(bool on) { set_lossless_output(on); }
|
||||
|
||||
void set_overflow_callback(NodeEventCallback cb) { event_callbacks_[0] = std::move(cb); }
|
||||
void set_network_overflow_callback(NodeEventCallback cb) override { event_callbacks_[1] = std::move(cb); }
|
||||
void set_closed_callback(NodeEventCallback cb) { closed_callbacks_[0] = std::move(cb); }
|
||||
@@ -717,6 +743,7 @@ private:
|
||||
Obj& obj_;
|
||||
std::shared_ptr<IScheduler> scheduler_;
|
||||
std::string name_;
|
||||
bool lossless_{false};
|
||||
std::size_t fifo_capacity_;
|
||||
input_channels_t input_channels_;
|
||||
output_channels_t output_channels_{};
|
||||
|
||||
Reference in New Issue
Block a user