Added callbacks for node errors and fifo overflow

Add new doc system which should/might deploy to pages.

docs/channels.md

@@ -0,0 +1,55 @@
+# Channels
+
+A `Channel<T>` is a lock-free SPSC (single-producer, single-consumer) ring buffer with atomic wait/notify.
+
+## Semantics
+
+- **Bounded**: fixed capacity set at construction. Default is 5 items.
+- **Backpressure**: when full, `push()` throws `ChannelOverflowError` immediately — no blocking, no spin.
+- **Blocking consumer**: `pop()` blocks until an item is available or the channel is disabled.
+- **Disable**: `channel.disable()` stops accepting pushes and unblocks any waiting `pop()` with `ChannelClosedError`.
+
+## Storage policy
+
+Small trivially-copyable types (≤ 8 bytes) are stored by value. Larger types are heap-allocated and passed via `shared_ptr<const T>` — one allocation per push, zero-copy fan-out:
+
+```cpp
+--8<-- "examples/04_storage_policy/main.cpp:storage_policy_spec"
+```
+
+Specialize `kpn::ChannelDataSize<T>` for accurate bandwidth reporting on heap-owning types:
+
+```cpp
+template<>
+struct kpn::ChannelDataSize<cv::Mat> {
+    static std::size_t bytes(const cv::Mat& m) { return m.total() * m.elemSize(); }
+};
+```
+
+## Named ports
+
+`in<"name">` and `out<"name">` tag nodes for readable wiring:
+
+```cpp
+--8<-- "examples/02_named_ports/main.cpp:named_port_creation"
+```
+
+Named ports are checked at compile time — a typo in a port name is a compile error.
+
+## Capacity tuning
+
+Set capacity per node at construction:
+
+```cpp
+auto node = make_node<my_func>(/*capacity=*/20);
+```
+
+Capacity is rounded up internally to the next power of two. Monitor fill levels via diagnostics to tune for your workload — a too-small capacity causes overflows; a too-large one wastes memory and hides producer/consumer speed mismatches.
+
+## Spin count
+
+`Channel` spins for up to ~4 µs (200 `pause` hints at ~20 ns each on x86) before sleeping on a futex. Set to 0 for power-constrained or predominantly-idle pipelines:
+
+```cpp
+Channel<int> ch(/*capacity=*/5, /*spin_count=*/0);
+```

docs/error-handling.md

@@ -0,0 +1,84 @@
+# Error Handling & Events
+
+KPN++ provides three complementary layers for observing and reacting to failures.
+
+---
+
+## 1. Per-node error handler
+
+Called when a node's function throws an unhandled exception. Return `true` to skip the failed invocation and keep running; `false` to stop the node.
+
+```cpp
+--8<-- "examples/15_node_error_handler/main.cpp:error_handler"
+```
+
+When a node stops (either from `false` return or no handler installed), it:
+
+1. Disables its **input** channels — upstream stops pushing into dead queues.
+2. Disables its **output** channels — downstream nodes receive `ChannelClosedError` on their next pop, propagating the shutdown naturally through the graph.
+
+---
+
+## 2. Per-node overflow callback
+
+Fired with a timestamp each time an output push is dropped because the channel is full. The node name is known at registration so it is not included — keeping the callback zero-overhead when unused.
+
+```cpp
+--8<-- "examples/16_event_callbacks/main.cpp:per_node_callback"
+```
+
+!!! note
+    The callback is purely informational — the node always continues after an overflow. To stop the node on overflow, call `node.stop()` from inside the callback.
+
+A matching `set_closed_callback()` fires (also with just a timestamp) when the node stops due to a closed upstream channel:
+
+```cpp
+node.set_closed_callback([](std::chrono::steady_clock::time_point ts) {
+    std::cerr << "node stopped at t=" << ts.time_since_epoch().count() << '\n';
+});
+```
+
+Each node holds two callback slots per event type — one user-set (registered above) and one injected by the network (see below). Both fire independently.
+
+---
+
+## 3. Network-level event handler
+
+One callback for the whole network. Receives the node name (captured in a closure by the network at `build()` / `start()`), a `NodeEvent`, and a timestamp:
+
+```cpp
+--8<-- "examples/16_event_callbacks/main.cpp:network_event_handler"
+```
+
+`NodeEvent` values:
+
+| Value | Meaning |
+|---|---|
+| `NodeEvent::Overflow` | An output push was dropped (channel full) |
+| `NodeEvent::Closed` | The node stopped (crash or upstream close cascade) |
+
+The network handler and any per-node callbacks are **independent** — both fire when set.
+
+---
+
+## Complete example
+
+`examples/16_event_callbacks/main.cpp` shows a fast producer overflowing a slow consumer, with both a per-node overflow callback and a network-level event handler active simultaneously.
+
+Node functions:
+
+```cpp
+--8<-- "examples/16_event_callbacks/main.cpp:node_fns"
+```
+
+Per-node overflow callback:
+
+```cpp
+--8<-- "examples/16_event_callbacks/main.cpp:per_node_callback"
+```
+
+Network-level event handler:
+
+```cpp
+--8<-- "examples/16_event_callbacks/main.cpp:network_event_handler"
+```

docs/examples.md

@@ -0,0 +1,40 @@
+# Examples
+
+All C++ examples are built by default and registered as CTest smoke tests. Run them all with:
+
+```bash
+ctest --test-dir build -L examples
+```
+
+## Index
+
+| Example | What it shows |
+|---|---|
+| `01_hello_pipeline` | Linear pipeline, index-based port wiring |
+| `02_named_ports` | `in<>`/`out<>` name tags, named port access |
+| `03_multi_output` | Tuple-returning node, per-element routing |
+| `04_storage_policy` | `channel_storage_policy` specialisation |
+| `05_error_handling` | Diagnostics handler, overflow channel stats |
+| `06_watchdog` | Watchdog interval, stall detection |
+| `10_static_hello_pipeline` | `StaticNetwork` + `make_network()` |
+| `11_static_fanout` | `StaticNetwork` with `FanoutNode` |
+| `15_node_error_handler` | `set_error_handler()` — skip or stop on exception |
+| `16_event_callbacks` | `set_overflow_callback()`, `set_event_handler()` |
+
+## OpenCV examples (optional)
+
+Built only when OpenCV ≥ 4 is found:
+
+| Example | What it shows |
+|---|---|
+| `09_opencv_cellshade` | Real-time cell-shading on webcam; `MainThreadNode` for display |
+| `12_static_cellshade` | Same pipeline as a `StaticNetwork` |
+| `13_debug_cellshade` | Web debug UI overlay on the cell-shading pipeline |
+
+Run the cell-shading example:
+
+```bash
+./build/examples/09_opencv_cellshade
+# Press 'q' or close the window to stop.
+# Falls back to an animated synthetic pattern if no webcam is found.
+```

docs/fanout.md

@@ -0,0 +1,44 @@
+# Fan-out & Routing
+
+## FanoutNode
+
+Reads one item and pushes a copy to each of N output channels. All downstream nodes receive every item.
+
+```cpp
+auto fan = make_fanout<Image, 2>(/*capacity=*/8);
+
+net.connect("src",  src.output<0>(),  "fan",   fan.input<0>())
+   .connect("fan",  fan.output<0>(),  "nodeA", nodeA.input<0>())
+   .connect("fan",  fan.output<1>(),  "nodeB", nodeB.input<0>());
+```
+
+If one downstream channel overflows, that output drops the item independently — the other outputs are unaffected.
+
+See `examples/11_static_fanout`.
+
+## RouterNode
+
+Reads one item and pushes it to exactly one of N outputs, chosen by a selector function:
+
+```cpp
+auto router = make_router<Frame, 3>(
+    [](const Frame& f) -> std::size_t { return f.stream_id % 3; });
+
+net.connect("src",    src.output<0>(),     "router", router.input<0>())
+   .connect("router", router.output<0>(),  "nodeA",  nodeA.input<0>())
+   .connect("router", router.output<1>(),  "nodeB",  nodeB.input<0>())
+   .connect("router", router.output<2>(),  "nodeC",  nodeC.input<0>());
+```
+
+If the selector returns `>= N` the item is silently dropped.
+
+## FilterNode
+
+Reads one item and passes it downstream only when a predicate returns `true`:
+
+```cpp
+auto filt = make_filter<Frame>([](const Frame& f) { return f.valid; });
+
+net.connect("src",  src.output<0>(),  "filt", filt.input<0>())
+   .connect("filt", filt.output<0>(), "dst",  dst.input<0>());
+```

docs/getting-started.md

@@ -0,0 +1,76 @@
+# Getting Started
+
+## Requirements
+
+| Dependency | Version | Notes |
+|---|---|---|
+| CMake | ≥ 3.21 | |
+| C++ compiler | GCC ≥ 11, Clang ≥ 13 | C++20 required |
+| nanobind | ≥ 2.1 | auto-fetched; Python ≥ 3.8 |
+| Catch2 | v3 | auto-fetched for tests |
+| OpenCV | ≥ 4 | optional; only for examples 09/12/13 |
+
+## Build
+
+```bash
+cmake -B build                          # core + tests + C++ examples
+cmake --build build --parallel
+ctest --test-dir build                  # run all tests including example smoke tests
+```
+
+Enable Python bindings:
+
+```bash
+cmake -B build -DKPN_BUILD_PYTHON=ON
+cmake --build build --parallel
+```
+
+Skip examples:
+
+```bash
+cmake -B build -DKPN_BUILD_EXAMPLES=OFF
+```
+
+## Your first pipeline
+
+Three functions — source, transform, sink — wired into a `Network`:
+
+```cpp
+--8<-- "examples/01_hello_pipeline/main.cpp:basic_node_fns"
+```
+
+Create nodes, connect them, build and run:
+
+```cpp
+--8<-- "examples/01_hello_pipeline/main.cpp:network_build"
+```
+
+That's it. Types are inferred from function signatures. The channel between `src` and `dbl` carries `int`; the channel between `dbl` and `prn` also carries `int`. A type mismatch is a compile error.
+
+## Named ports
+
+For nodes with multiple inputs or outputs, name the ports for clarity:
+
+```cpp
+--8<-- "examples/02_named_ports/main.cpp:named_port_creation"
+```
+
+Wire by name instead of index:
+
+```cpp
+--8<-- "examples/02_named_ports/main.cpp:named_port_network"
+```
+
+## Multi-output nodes
+
+Return a `std::tuple` to fan out to multiple downstream nodes:
+
+```cpp
+--8<-- "examples/03_multi_output/main.cpp:multi_output_fn"
+```
+
+Wire each tuple element to its own downstream node:
+
+```cpp
+--8<-- "examples/03_multi_output/main.cpp:fanout_network"
+```

docs/index.md

@@ -0,0 +1,39 @@
+# KPN++
+
+A C++20 [Kahn Process Network](https://en.wikipedia.org/wiki/Kahn_process_networks) library. Each node wraps a plain function and runs concurrently, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind.
+
+---
+
+## Why KPN++?
+
+- **Zero boilerplate** — wrap any callable as a node; types flow automatically from the function signature
+- **Bounded channels** — backpressure is structural, not bolted on
+- **Observable** — per-node and network-level callbacks for overflow and stop events; diagnostics snapshots; optional web UI
+- **Composable** — `Network` for runtime wiring, `StaticNetwork` for compile-time topology with zero overhead
+
+---
+
+## Quick example
+
+```cpp
+#include <kpn/kpn.hpp>
+using namespace kpn;
+
+--8<-- "examples/01_hello_pipeline/main.cpp:basic_node_fns"
+
+int main() {
+--8<-- "examples/01_hello_pipeline/main.cpp:network_build"
+}
+```
+
+---
+
+## Install & build
+
+```bash
+cmake -B build
+cmake --build build --parallel
+ctest --test-dir build          # unit tests + example smoke tests
+```
+
+See [Getting Started](getting-started.md) for full build options.

docs/network.md

@@ -0,0 +1,67 @@
+# Networks
+
+A `Network` wires nodes together at runtime using a builder chain.
+
+## Building a network
+
+```cpp
+--8<-- "examples/01_hello_pipeline/main.cpp:network_build"
+```
+
+The builder chain:
+
+| Method | Purpose |
+|---|---|
+| `.add(name, node)` | Register a node; assigns its name |
+| `.connect(src, port, dst, port)` | Wire one output port to one input port |
+| `.build()` | Compute topological order; inject network callbacks |
+| `.start()` | Start nodes in topological order |
+| `.stop()` | Stop all nodes immediately |
+| `.shutdown()` | Graceful drain: stop sources first, wait for channels to empty, then stop downstream |
+
+## Port access
+
+Ports are accessed by index or by name:
+
+```cpp
+// By index
+net.connect("src", src.output<0>(), "dst", dst.input<0>());
+
+// By name (requires named ports)
+--8<-- "examples/02_named_ports/main.cpp:named_port_network"
+```
+
+## Diagnostics
+
+Install a diagnostics handler to receive periodic snapshots of every node and channel:
+
+```cpp
+--8<-- "examples/05_error_handling/main.cpp:diagnostics_handler"
+```
+
+Or print a full report at any time:
+
+```cpp
+net.print_diagnostics();   // writes to stderr by default
+net.print_diagnostics(std::cout);
+```
+
+## Network-level event handler
+
+Observe overflow and node-stop events across the entire network in one place:
+
+```cpp
+--8<-- "examples/16_event_callbacks/main.cpp:network_event_handler"
+```
+
+`NodeEvent` is either `NodeEvent::Overflow` (item dropped on full channel) or `NodeEvent::Closed` (node stopped due to crash or closed upstream channel). See [Error Handling & Events](error-handling.md).
+
+## Shutdown
+
+`net.stop()` halts immediately — all nodes stop in reverse topological order.
+
+`net.shutdown()` drains gracefully: source nodes stop first; their output channels are polled until empty; then the next layer stops, and so on. This ensures no items are lost if downstream nodes are still consuming.
+
+## StaticNetwork
+
+For zero-overhead compile-time topology, see [Static Networks](static-network.md).

docs/nodes.md

@@ -0,0 +1,81 @@
+# Nodes
+
+A node wraps any callable. Its input types are inferred from the function's parameter list; its output types from the return type.
+
+## Node types
+
+| Type | Thread model | Use case |
+|---|---|---|
+| `Node<Func>` | Dedicated thread per node | Default — simplest, most isolated |
+| `PoolNode<Func>` | Shared `ThreadPool` | Many nodes, resource-bounded execution |
+| `InterruptNode<Func>` | Event-driven, no thread | Camera frame ready, timer tick, socket |
+| `FanoutNode<T, N>` | Dedicated thread | Broadcast one item to N outputs |
+| `RouterNode<T, N>` | Dedicated thread | Route one item to one of N outputs |
+| `FilterNode<T>` | Dedicated thread | Pass items matching a predicate |
+
+## Creating nodes
+
+All node types are created via factory functions that infer types from the callable:
+
+```cpp
+// Free function — simplest case
+auto node = make_node<my_func>();
+
+// Stateful functor (operator() is the function)
+MyProcessor proc;
+auto node = make_node(proc);
+
+// Pool node — shares a ThreadPool with other nodes
+auto pool = std::make_shared<ThreadPool>(4);
+auto node = make_pool_node<my_func>(pool);
+
+// Interrupt node — triggered externally
+auto sched = std::make_shared<ThreadPool>(2);
+auto node  = make_interrupt_node<produce_frame>(sched, out<"frame">{});
+camera_sdk.on_frame_ready(node.get_trigger());
+```
+
+## Channel capacity
+
+Each node's input FIFO has a configurable capacity (default 5):
+
+```cpp
+auto node = make_node<my_func>(/*capacity=*/20);
+auto node = make_pool_node<my_func>(pool, /*capacity=*/20);
+```
+
+When an upstream push would exceed capacity, `ChannelOverflowError` is thrown and the item is dropped. See [Error Handling & Events](error-handling.md) to observe and react to this.
+
+## Source nodes
+
+A node with no inputs is a source. It self-submits immediately on `start()` and re-submits after each execution:
+
+```cpp
+static int produce() {
+    std::this_thread::sleep_for(std::chrono::milliseconds(10));
+    return ++counter;
+}
+auto src = make_node<produce>();
+```
+
+!!! tip
+    Source nodes must sleep or yield to avoid overflowing their output channel. The channel capacity provides the only bound.
+
+## Sink nodes
+
+A node with a `void` return is a sink — it consumes items without producing output:
+
+```cpp
+static void print_it(int x) { std::cout << x << '\n'; }
+auto snk = make_node<print_it>();
+```
+
+## Error handler
+
+When a node's function throws an unhandled exception, the default behaviour is to stop the node (disabling its channels so the shutdown cascades downstream). Install a handler to override:
+
+```cpp
+--8<-- "examples/15_node_error_handler/main.cpp:error_handler"
+```
+
+See [Error Handling & Events](error-handling.md) for the full picture.

docs/requirements.txt

@@ -0,0 +1,3 @@
+mkdocs>=1.5
+mkdocs-material>=9.5
+pymdown-extensions>=10.0

docs/shared-resource.md

@@ -0,0 +1,42 @@
+# Shared Resources
+
+`SharedResource<T>` arbitrates exclusive access to a resource (ONNX session, CUDA stream, serial port) across multiple nodes using a priority-based waiter queue with starvation prevention.
+
+## Usage
+
+```cpp
+#include <kpn/shared_resource.hpp>
+using namespace kpn;
+
+SharedResource<OnnxSession> model(session_args...);
+
+static cv::Mat run_inference(cv::Mat frame) {
+    // Acquires the model; releases automatically on scope exit.
+    auto guard = model.acquire_balanced(in_channel, out_channel);
+    return guard->Run(frame);
+}
+```
+
+## Acquire modes
+
+| Method | Priority |
+|---|---|
+| `acquire()` | Equal (fair FIFO) |
+| `acquire(fn)` | Custom — `fn()` returns `float` in `[0, 1]` |
+| `acquire_balanced(in_ch, out_ch)` | `input_fill × output_headroom` — highest urgency wins |
+
+`acquire_balanced` favours nodes with full input queues and empty output queues — the node that has the most work to do and nowhere to stall wins the resource next.
+
+## Starvation prevention
+
+Each waiter's effective score grows with elapsed wait time (`0.05` per second by default), ensuring a low-priority node eventually gets served regardless of how frequently higher-priority nodes compete.
+
+## Diagnostics
+
+Register with the network for snapshot reporting:
+
+```cpp
+net.register_resource("model", &model);
+```
+
+The diagnostics table then shows acquisition count, mean wait time, and current waiter count.

docs/static-network.md

@@ -0,0 +1,46 @@
+# Static Networks
+
+`StaticNetwork` encodes the entire topology at compile time using a `make_network()` builder. Nodes and channel types are verified statically with zero runtime overhead.
+
+## Usage
+
+```cpp
+#include <kpn/kpn.hpp>
+using namespace kpn;
+
+static int  produce()        { return 42; }
+static int  double_it(int x) { return x * 2; }
+static void print_it(int x)  { std::cout << x << '\n'; }
+
+int main() {
+    auto src = make_node<produce>  ();
+    auto dbl = make_node<double_it>();
+    auto prn = make_node<print_it> ();
+
+    auto net = make_network(
+        edge(src, src.output<0>(), dbl, dbl.input<0>()),
+        edge(dbl, dbl.output<0>(), prn, prn.input<0>())
+    );
+
+    net.set_event_handler([](std::string_view name, NodeEvent ev, auto ts) {
+        // same API as Network
+    });
+
+    net.start();
+    std::this_thread::sleep_for(std::chrono::milliseconds(100));
+    net.stop();
+}
+```
+
+See `examples/10_static_hello_pipeline` and `examples/11_static_fanout`.
+
+## When to use
+
+| | `Network` | `StaticNetwork` |
+|---|---|---|
+| Topology known at | Runtime | Compile time |
+| Type checking | Runtime (`dynamic_cast`) | Compile time |
+| Overhead | Minimal | Zero |
+| Flexibility | Add nodes dynamically | Fixed at compile time |
+
+For most applications `Network` is sufficient. Use `StaticNetwork` when you need the absolute minimum overhead or want compile-time topology verification.