Commit Graph
5 Commits
Author SHA1 Message Date
dtourolleandClaude Opus 5 ac36159f37 fix: apply lossless push in PoolObjectNode too
PoolNode and PoolObjectNode each have their own push_one_out(). The previous
commit only reached PoolNode's copy, so ObjectNode-based pipelines — the common
case — still dropped on overflow with lossless enabled.

Verified end to end: a 2300-frame source now reports exactly 2300 frames at
every stage, where it previously delivered 774 to the second node.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-25 23:12:51 +02:00
dtourolleandClaude Opus 5 4e81752838 feat: opt-in lossless (blocking) output for nodes and fanouts
Channels drop on overflow by default. That is the right behaviour for live
sources, where a stale item is worth less than a fresh one, but it makes the
library unusable for offline batch work: items vanish with no diagnostic, and
any downstream analysis that assumes a fixed sample rate is silently invalid.

Auto-inserted fanouts were the harder half of this. They are created inside
make_network(), so user code cannot reach them to configure, and they drop
per-output inside a swallowed catch — so a pipeline whose own nodes were all
configured lossless could still lose items with nothing reported anywhere. In
the pipeline this came from, capture's 2505 frames arrived at the detector as
774 while the overflow counter read zero.

Adds:
  - INode::set_lossless_output(bool), defaulted to a no-op so node types with
    no output channels ignore it
  - PoolNode / PoolObjectNode: route push_one_out() through push_blocking()
  - FanoutNode: same, plus set_lossy_output(i) to opt a single branch back out
  - StaticNetwork::set_lossless(), which reaches user nodes and fanouts alike
  - StaticNetwork::drain(), a public wrapper over the existing private
    drain_all_channels(), so callers can flush in-flight work before stop()

Default behaviour is unchanged; every path is off unless explicitly enabled.

Blocking output is only safe when every consumer eventually drains. A branch
that can stall indefinitely — a display node nobody is servicing — will apply
backpressure to the whole pipeline, which is what set_lossy_output() is for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-25 23:03:56 +02:00
dtourolleandClaude Opus 4.8 19f5a2b0ae Deliver EOF sentinels out-of-band to prevent teardown deadlock
🚦 CI / changes (push) Successful in 18s
🚦 CI / docker (push) Has been skipped
🚦 CI / docs (push) Has been cancelled
🚦 CI / test (push) Has been cancelled
Channel::push() drops values on overflow (the intended backpressure
policy for data), and PoolNode swallows the resulting ChannelOverflowError.
For a control sentinel like EOF this is fatal: a single dropped EOF under
backpressure wedges every downstream pop() forever, so the pipeline never
tears down.

Deliver sentinels out-of-band instead. Channel::push_sentinel() stores the
token in a dedicated slot that does not consume ring capacity, so it can
never overflow and — crucially — never blocks the caller. That non-blocking
property is essential: each KPN node has a single worker thread, so a
*blocking* push would park that thread and stop it draining its own input,
cascading into a hold-and-wait deadlock under backpressure. The consumer's
pop()/try_pop_now() drain the ring first, then deliver the sentinel, so it
always arrives after every value pushed before it.

approx_size() (which node readiness checks call) counts a pending sentinel
as consumable work, so a channel carrying only a sentinel still schedules
its consumer's next fire — without this the token would sit undelivered and
the pipeline would still deadlock at teardown.

PoolNode/PoolObjectNode route values carrying an eof flag (direct .eof or
nested .source.eof) through push_sentinel via a SFINAE-safe is_sentinel_value
trait; all other values keep the existing lossy throwing push. The trait
compiles to false for types without an eof convention, so this is a no-op
for pipelines that don't use one.

Verified end-to-end: scene_analyze now reaches EOF, flushes its output, and
exits cleanly instead of hanging.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 00:30:40 +02:00
dtourolle 6f384dc4b5 Added callbacks for node errors and fifo overflow
📚 Docs / deploy (push) Failing after 7s
🧪 Test / test (push) Has been cancelled
Add new doc system which should/might deploy to pages.
2026-06-19 22:26:39 +02:00
dtourolle f6bcaa15b0 Performance improvements, better readme and complete python bindings
🧪 Test / test (push) Failing after 28m30s
2026-05-12 21:23:33 +02:00