# KPN++ — Kahn Process Network Library Specification

## Overview

A C++20 template-metaprogramming library for building Kahn Process Networks, where each node
wraps a function/method, runs in its own thread, and communicates via bounded FIFO queues.
Includes nanobind bindings for Python graph construction and prototyping.

---

## Project Structure

```
kpn++/
├── CMakeLists.txt
├── include/kpn/
│   ├── fixed_string.hpp    # NTTP string type for named ports
│   ├── traits.hpp          # Function signature introspection
│   ├── channel.hpp         # Bounded FIFO channel + storage policy
│   ├── node.hpp            # Node wrapper + thread management
│   ├── port.hpp            # Input/Output port handles
│   ├── network.hpp         # Graph builder + orchestrator/watchdog
│   ├── variant_node.hpp    # Runtime-typed node for Python graphs
│   └── python/
│       └── bindings.hpp    # Nanobind binding helpers
├── src/
│   └── network.cpp         # Orchestrator thread impl
├── tests/
├── examples/
│   ├── 01_hello_pipeline/
│   ├── 02_named_ports/
│   ├── 03_multi_output/
│   ├── 04_storage_policy/
│   ├── 05_error_handling/
│   ├── 06_watchdog/
│   ├── 07_python_network/
│   ├── 08_python_subport/
│   └── 09_opencv_cellshade/    # optional, requires OpenCV
└── python/
    └── kpn_python.cpp      # Nanobind module definition
```

---

## Component 0 — `fixed_string.hpp`: NTTP String

Named ports use C++20 non-type template parameters (NTTPs). `std::string_view` and
`const char*` are not valid NTTPs because they are not structurally comparable. The standard
solution is a `fixed_string` literal type with `constexpr` internal storage.

```cpp
template<std::size_t N>
struct fixed_string {
    char data[N]{};
    constexpr fixed_string(const char (&s)[N]) { std::copy_n(s, N, data); }
    constexpr bool operator==(const fixed_string&) const = default;
    constexpr std::string_view view() const { return {data, N - 1}; }
};

// Deduction guide — required so fixed_string("img") works as an NTTP.
// Without it the compiler cannot infer N and the named-port API does not compile.
template<std::size_t N>
fixed_string(const char (&)[N]) -> fixed_string<N>;
```

`fixed_string<3>` and `fixed_string<6>` are distinct types, so `input<"img">()` and
`input<"sigma">()` produce different template instantiations — this is intentional and
enables zero-overhead compile-time port dispatch.

Named-port lookup uses a `constexpr` function over the name pack. It returns a sentinel
`npos` on miss rather than `static_assert`-ing internally, so the assertion fires at the
`input<"img">()` call site — giving the user a readable error at the point of use instead
of deep in template instantiation:

```cpp
inline constexpr std::size_t npos = std::size_t(-1);

template<fixed_string Name, fixed_string... Names>
constexpr std::size_t index_of() {
    std::size_t i = 0;
    bool found = false;
    ((Name == Names ? (found = true) : (found ? 0 : ++i)), ...);
    return found ? i : npos;
}

// Used at the call site:
template<fixed_string N>
auto input() {
    constexpr std::size_t idx = index_of<N, InputNames...>();
    static_assert(idx != npos, "unknown input port name");
    return input<idx>();
}
```

---

## Component 1 — `traits.hpp`: Function Introspection

Extracts parameter types and return type from any callable at compile time.

```cpp
// For: Image blur(Image in, float sigma)
// function_traits<decltype(blur)>::args     == std::tuple<Image, float>
// function_traits<decltype(blur)>::return_t == Image

// For multi-output: std::tuple<Image, Mask> detect(Image in)
// return_t == std::tuple<Image, Mask>  →  2 output ports
// return_t == Image                    →  1 output port (normalised to tuple<Image> internally)
// return_t == void                     →  0 output ports (sink node)
```

Handles: free functions, lambdas, `std::function`, member function pointers.

A helper alias normalises the return type to always be a tuple for uniform handling in
`run_loop`:

```cpp
template<typename T>
using normalised_return_t =
    std::conditional_t<is_tuple_v<T>, T, std::tuple<T>>;
// void return → std::tuple<> (empty tuple, zero output ports)
```

---

## Component 2 — `channel.hpp`: Bounded FIFO + Storage Policy

### Storage Policy

The type stored in a channel depends on a specialisable trait. Users can override it for
any type:

```cpp
template<typename T>
struct channel_storage_policy {
    static constexpr bool by_value =
        std::is_trivially_copyable_v<T> && sizeof(T) <= 8;
};

// User opt-in to value semantics for a small struct:
template<> struct channel_storage_policy<MySmallStruct> {
    static constexpr bool by_value = true;
};

// Derived storage type:
template<typename T>
using channel_storage_t = std::conditional_t<
    channel_storage_policy<T>::by_value,
    T,
    std::shared_ptr<const T>
>;
```

### Channel

`Channel<T>` stores `channel_storage_t<T>` internally. The producer calls `push(T value)`
and the channel transparently wraps it in `make_shared<const T>` when needed. All consumers
of the same channel receive the same `shared_ptr` — no copies of large objects.

`run_loop` dereferences `shared_ptr<const T>` before passing to the wrapped function, so a
function declared `void f(const Image& img)` works naturally and the compiler enforces
immutability — no policy enforcement or `const_cast` needed.

```cpp
template<typename T>
class Channel {
public:
    using storage_type = channel_storage_t<T>;

    explicit Channel(std::size_t capacity = 5);

    void push(T value);        // wraps in shared_ptr<const T> if needed; throws on overflow
    T    pop();                // blocks (KPN semantics); unwraps shared_ptr if needed
    bool try_pop(T& out, std::chrono::milliseconds timeout);

    std::size_t size()     const;
    std::size_t capacity() const;
};

class ChannelOverflowError : public std::runtime_error {};
```

### Ownership

A `Channel<T>` is **owned by its consumer node** — it lives as a member of the destination
node. The producer node holds a non-owning raw pointer to push into it. The channel is
destroyed when its consumer is destroyed, which is the correct lifetime.

The `Network` itself is **non-owning** — nodes are declared by the user and outlive the
network. `net.add("name", node)` registers a raw pointer; the user is responsible for keeping
nodes alive for the network's lifetime. This avoids type-erasure ownership complexity and
keeps node construction explicit.

### Backpressure and Shutdown — `accepting_` Flag

Each channel carries a single `std::atomic<bool> accepting_` (default `true`). This is the
**sole shutdown mechanism** — no `try_pop` polling, no sentinel values, no drain logic.

```cpp
template<typename T>
class Channel {
    std::atomic<bool> accepting_{true};
public:
    void push(T value) {
        if (!accepting_.load(std::memory_order_relaxed)) return; // silently drop
        // normal push — throws ChannelOverflowError if full
    }

    void enable()  { accepting_.store(true,  std::memory_order_relaxed); }
    void disable() {
        accepting_.store(false, std::memory_order_relaxed);
        clear();        // drop all queued items immediately
        cv_.notify_all(); // unblock any waiting pop()
    }
};
```

**Who flips the flag:** the **consumer node** — it's the channel owner. `node.stop()` calls
`disable()` on all its input channels. `node.start()` calls `enable()`. The producer never
touches the flag; it calls `push()` and if the channel is disabled the value is silently
dropped and the producer continues.

**Overflow** (`push()` on a full, accepting channel) still throws `ChannelOverflowError` —
this signals a design error (undersized FIFO) and is unchanged.

**Blocking `pop()`** unblocks immediately when `disable()` is called (via `cv_.notify_all()`),
and throws `ChannelClosedError` if the queue is empty and the channel is disabled.

### `try_pop` Purpose

`try_pop` exists for **watchdog polling only** — not for shutdown (the `accepting_` flag
handles that) and not for normal processing. The watchdog uses it to probe whether a node
is making progress without blocking the watchdog thread.

> **Note (C++ compile-time graphs):** In a fully compiled C++ graph the variant never
> appears. The compiler wires `Channel<Image>` to `Channel<Image>` directly. The variant is
> a pure compile-time construct used only for type-checking and generates zero runtime
> overhead.

---

## Component 3 — `port.hpp`: Port Handles

```cpp
template<typename NodeT, std::size_t Idx>
struct InputPort  { NodeT& node; };

template<typename NodeT, std::size_t Idx>
struct OutputPort { NodeT& node; };
```

Nodes expose port handles via:

```cpp
// By index — always available
node_a.input<0>()    // returns InputPort<NodeA, 0>
node_b.output<1>()   // returns OutputPort<NodeB, 1>

// By name — only valid when names were provided at make_node time
node_a.input<"img">()
node_b.output<"edges">()
```

Named access resolves to an index at compile time via `index_of` (see `fixed_string.hpp`).
Zero runtime cost — the name dispatch is fully eliminated by the compiler.

---

## Component 4 — `node.hpp`: Node Wrapper

```cpp
template<
    auto Func,
    fixed_string... InputNames,   // optional; count must match arity or be 0
    fixed_string... OutputNames   // optional; count must match output count or be 0
>
class Node {
public:
    explicit Node(std::size_t fifo_capacity = 5);

    void start();
    void stop();   // signals thread to finish current item then exit

    // Port access — by index
    template<std::size_t I>  auto input();
    template<std::size_t I>  auto output();

    // Port access — by name (compile error if names were not provided)
    template<fixed_string N> auto input();
    template<fixed_string N> auto output();

    static constexpr std::size_t input_count;
    static constexpr std::size_t output_count;

private:
    void run_loop();
    // Pops each input channel, dereferences shared_ptr if needed,
    // calls Func, unpacks the normalised tuple return,
    // pushes each element to its output channel.

    std::thread thread_;
    // Input channels owned here (one per input port).
    // Output channel pointers (non-owning) set at connect time.
};
```

**Factory syntax — `in<>` / `latch<>` / `out<>` tag structs:**

A flat name pack `make_node<f, "a", "b", "c">` is ambiguous (where do inputs end?).
Option chosen: `in<...>`, `latch<...>`, and `out<...>` tag types that wrap the name packs unambiguously.
All are optional; omitting either means those ports are index-only.

```cpp
// Tag types (trivial, no data):
template<fixed_string... Names> struct in    {};
template<fixed_string... Names> struct latch {};
template<fixed_string... Names> struct out   {};

// Factory:
// No names
auto node = make_node<my_func>(/*fifo_capacity=*/10);

// Input names only
auto node = make_node<my_func, in<"img","sigma">>(10);

// Both input and output names
auto node = make_node<my_func, in<"img","sigma">, out<"blurred","mask">>(10);

// Mixed synchronous and latched inputs
auto node = make_node<my_func, in<"error">, latch<"setpoint">, out<"output">>(10);
```

**Wrong name count is a compile error.** The `Node` class `static_assert`s that
`sizeof...(InputNames) == 0 || sizeof...(InputNames) == input_count` (and same for outputs).
Without this, a mismatch between name count and arity produces an unreadable template error.

```cpp
static_assert(
    sizeof...(InputNames) == 0 || sizeof...(InputNames) == input_count,
    "make_node: number of input names must match function arity, or provide none"
);
```

Multi-output functions must return `std::tuple<...>`. Single return accepted as-is.
`void` return = sink node (no output ports).

---

## Component 4a — Latched Input Ports

### Motivation

Control and robotics applications naturally have two kinds of inputs at different update rates:

- **Synchronous inputs** (`in<>`) — the node must have fresh data on every fire. Typical for sensor readings that drive the computation (e.g. encoder RPM).
- **Latched inputs** (`latch<>`) — the node uses the most recently received value, and does not block if no new value has arrived. Typical for setpoints or parameters that change infrequently relative to the control loop (e.g. bearing from a CV pipeline, PID gains).

Without latched ports, a node must block on all inputs simultaneously. This forces the control loop to run at the rate of the slowest input — unacceptable when a 1kHz encoder loop must wait for a 30Hz vision update.

### Semantics

A `latch<>` port:

1. **Does not block** if its channel is empty — it reuses the last successfully popped value.
2. **Does block on first fire** — there is no meaningful "default" value, so the node waits until at least one value has arrived on each latched port before firing for the first time.
3. **Consumes the value** when one is available (standard `pop()`), then holds it until the next value arrives.

The node fires whenever all `in<>` ports have data, using the last known value for each `latch<>` port.

### Implementation in `run_loop`

`run_loop` maintains a `std::tuple` of cached values, one slot per latched port. On each iteration:

```cpp
// Synchronous ports — blocking pop (existing behaviour)
auto sync_args = std::make_tuple(input<0>().pop(), input<1>().pop(), ...);

// Latched ports — non-blocking try_pop; keep cached value on miss
try_pop(latch_cache_<I>, latch_channel_<I>);   // updates cache if data available

// Call wrapped function with merged argument tuple
auto result = std::apply(Func, merge(sync_args, latch_cache_));
```

Latched channels are otherwise identical to synchronous channels: bounded FIFO, `shared_ptr` storage policy, same shutdown behaviour.

### Example — PID with live setpoint

```cpp
// bearing arrives at ~30 Hz from CV; rpm arrives at ~1 kHz from encoder
double pid_compute(double rpm, double bearing) { ... }

auto pid = make_node<pid_compute,
    in<"rpm">,          // synchronous — blocks until fresh encoder tick
    latch<"bearing">,   // latched — uses last known bearing from CV
    out<"pwm">
>(8);

Network net;
net.add("tacho",   tacho_node)
   .add("tracker", tracker_node)
   .add("pid",     pid)
   .connect("tacho",   tacho_node.output<"rpm">(),     "pid", pid.input<"rpm">())
   .connect("tracker", tracker_node.output<"bearing">(),"pid", pid.input<"bearing">())
   .build();
```

The PID node fires at encoder rate. If no new bearing has arrived since the last tick, it reuses the previous one — correct behaviour for a control loop.

### Port Ordering Contract

`in<>` and `latch<>` ports together must cover all function parameters in declaration order. The `static_assert` on name count is extended to cover both tags jointly:

```cpp
static_assert(
    sizeof...(InNames) + sizeof...(LatchNames) == input_count ||
    (sizeof...(InNames) == 0 && sizeof...(LatchNames) == 0),
    "make_node: in<> and latch<> names together must match function arity, or provide none"
);
```

The function parameter at position `i` is synchronous if `i` is in the `in<>` pack, latched if in the `latch<>` pack. Mixed ordering is allowed — the tag packs define which positions are latched, not a contiguous suffix.

---

## Component 5 — `network.hpp`: Graph Builder + Orchestrator

`Network` is **non-owning** — nodes are declared by the user and must outlive the network.
`add()` registers a raw pointer. Graph construction uses a builder pattern so the full
topology is known before `build()`, enabling cycle detection and topological ordering.

```cpp
class Network : public INode {   // Network is itself an INode — enables sub-networks
public:
    // Register a node by name. NodeT must satisfy INode. Network holds a raw pointer.
    template<typename NodeT>
    Network& add(std::string name, NodeT& node);

    // Connect output port of src to input port of dst.
    // Type mismatch → static_assert at compile time.
    template<typename SrcNode, std::size_t SrcIdx,
             typename DstNode, std::size_t DstIdx>
    Network& connect(const std::string& src_name, OutputPort<SrcNode, SrcIdx>,
                     const std::string& dst_name, InputPort<DstNode,  DstIdx>);

    // Expose an internal node's input/output as a boundary port of this (sub-)network.
    // Allows a Network to be connected into a larger Network like a single node.
    template<typename NodeT, std::size_t Idx>
    Network& expose_input(std::string boundary_name, InputPort<NodeT, Idx>);

    template<typename NodeT, std::size_t Idx>
    Network& expose_output(std::string boundary_name, OutputPort<NodeT, Idx>);

    // DFS cycle check + topological sort. Throws NetworkCycleError on cycles.
    Network& build();

    void start() override;   // starts all internal nodes in topological order
    void stop()  override;   // stops all internal nodes in reverse order; disables channels
    bool running() const override;

    void set_watchdog_interval(std::chrono::milliseconds);

    using ErrorHandler = std::function<void(std::string_view node_name, std::exception_ptr)>;
    void set_error_handler(ErrorHandler);

private:
    std::map<std::string, INode*>                   nodes_;      // non-owning
    std::map<std::string, std::vector<std::string>> adj_;
    std::vector<std::string>                        topo_;
    std::jthread                                    watchdog_;
    std::chrono::milliseconds                       watchdog_interval_{500};
    ErrorHandler                                    error_handler_;
};
```

**Node lifetime contract:** nodes must outlive the `Network`. The typical pattern is to
declare nodes and the network in the same scope:

```cpp
// Nodes declared first — they own their input channels
auto blur   = make_node<blur_func,   in<"img","sigma">>(10);
auto detect = make_node<detect_func, in<"img">>(10);

Network net;
net.add("blur",   blur)
   .add("detect", detect)
   .connect("blur",   blur.output<0>(),          "detect", detect.input<0>())
   .connect("blur",   blur.output<"blurred">(),  "detect", detect.input<"img">())
   .build();
net.start();
```

**Sub-networks** — because `Network` implements `INode`, it can be registered inside a
larger `Network` as a named node. Boundary ports declared via `expose_input` /
`expose_output` make the internal nodes' ports available to the outer graph:

```cpp
// Inner sub-network
auto stage1 = make_node<preprocess>(5);
auto stage2 = make_node<enhance>(5);
Network pipe;
pipe.add("pre", stage1).add("enh", stage2)
    .connect("pre", stage1.output<0>(), "enh", stage2.input<0>())
    .expose_input("img",    stage1.input<0>())
    .expose_output("result", stage2.output<0>())
    .build();

// Outer network treats `pipe` as a single node
auto sink = make_node<display>(5);
Network top;
top.add("pipe", pipe).add("sink", sink)
   .connect("pipe", pipe.output<"result">(), "sink", sink.input<0>())
   .build();
top.start();
```

`NetworkCycleError` is thrown by `build()` if the graph contains a directed cycle.

---

## Component 6 — `variant_node.hpp`: Runtime-typed Node (Python graphs)

### Motivation

Python graphs cannot use compile-time type resolution. A `PyNetwork` is constructed with a
**closed list of C++ node types** known at binding time. The library derives a deduplicated
`std::variant` from all port types across those nodes. Type safety is enforced at
`connect()` time via string signatures.

### Variant Deduplication

All port types from the registered nodes are collected into a flat pack, duplicates are
removed via a `unique_types` TMP metafunction, then the variant is instantiated once:

```cpp
template<typename... Nodes>
using py_variant_t = std::variant<unique_types_t<all_port_types_t<Nodes...>>>;
```

This is pure TMP and runs entirely at compile time. The resulting variant has no redundant
alternatives at runtime.

### PyNetwork Construction

`make_py_network` is a **pure C++ template** — no CMake code-gen step. The variant is
derived entirely at compile time from the registered node type list. The nanobind module
definition is the single place where node types are listed; recompiling the extension is
the "registration" step.

```cpp
// In kpn_python.cpp — list all node types that may appear in Python graphs:
auto py_net = make_py_network<NodeA, NodeB, NodeC>();
//   VariantValue = std::variant< /* deduplicated port types from A, B, C */ >
// Registers to_python / from_python converters for each alternative.
```

### VariantChannel

```cpp
using VariantValue = py_variant_t</* registered nodes */>;

class VariantChannel {
public:
    explicit VariantChannel(std::size_t capacity = 5);
    void push(VariantValue v);   // throws ChannelOverflowError if full
    VariantValue pop();          // blocks (KPN semantics)
};
```

### VariantNode

Wraps a registered C++ node type. Its `run_loop` uses `std::visit` to extract the concrete
type from a `VariantValue`, calls the underlying function, then wraps the result back into a
`VariantValue` for the output channel.

```cpp
class VariantNode {
public:
    std::string input_type_sig(std::size_t idx) const;
    std::string output_type_sig(std::size_t idx) const;

    void connect_input (std::size_t port, std::shared_ptr<VariantChannel>);
    void connect_output(std::size_t port, std::shared_ptr<VariantChannel>);

    void start();
    void stop();
};
```

### PythonConverter — Crossing the C++/Python Boundary

Every type in the variant must provide a `PythonConverter` specialisation. This is the
single mechanism used for all data crossing into or out of Python (PyNodes, `net.read`,
`net.write`):

```cpp
template<typename T>
struct PythonConverter {
    static nanobind::object to_python(const T&);
    static T                from_python(nanobind::object);
};
```

### PyNode — Pure Python Processing Node

A `PyNode` holds a `nanobind::object` as its function. Its `run_loop`:

1. Pops `VariantValue` from each input channel
2. `std::visit` → calls `PythonConverter<T>::to_python` for each → **acquires GIL**
3. Calls the Python callable
4. **Releases GIL** → calls `PythonConverter<R>::from_python` on the return value
5. Pushes result as `VariantValue` to output channel

### Sub-value Extraction and Injection

A C++ node returning `std::tuple<A, B, C>` exposes three independent output ports. Each
element is pushed to its own `VariantChannel` — sub-indexing is a first-class concept at the
channel level, not an afterthought.

**Python tap — read one output port into Python:**

```python
value = net.read("detect", output=2)
# Pops from output channel 2, calls PythonConverter<C>::to_python.
# GIL released while blocking on pop(), re-acquired before to_python call.
```

**Python inject — write a Python value into a specific input port:**

```python
net.write("blur", input=1, value=my_sigma)
# Calls PythonConverter<float>::from_python(my_sigma), pushes to input channel 1.
# GIL released while blocking on push() if channel is full.
```

**Python splitter node:**

```python
def split(packed):
    img, mask, score = packed
    return img, mask

net.add_node("split", split, inputs=["packed"], outputs=["img", "mask"])
net.connect("detect", 0, "split",  0)
net.connect("split",  0, "show",   0)
net.connect("split",  1, "save",   0)
```

**Direct C++ sub-output to Python node input:**

```python
net.connect("detect", 1, "py_thresh", 0)
# Type sig of detect:output[1] must match py_thresh:input[0] — checked at connect().
```

### Type-check at connect time (Python)

```python
# Raises kpn.TypeError if signatures don't match
net.connect("blur", 0, "thresh", 0)   # (src_name, out_idx, dst_name, in_idx)
```

---

## Component 7 — Orchestrator / Watchdog

Runs in its own dedicated thread inside `Network` / `PyNetwork`. Responsibilities:

- Starts nodes in topological order; stops them in reverse order
- Tracks per-node execution time (exponential moving average)
- Emits warning via logger callback (default: `stderr`) when a node stalls beyond threshold
- Catches exceptions from node threads and routes them to `ErrorHandler`
- Graceful shutdown: signals all nodes, joins with timeout, reports any that fail to stop

---

## GIL Rules (non-negotiable constraints on binding implementation)

Two rules govern all interaction between node threads and the Python interpreter:

1. **Acquire for callback** — a node thread must hold the GIL only for the duration of a
   Python callable invocation (`nb::gil_scoped_acquire` wrapping the call site).

2. **Release while blocking** — any blocking operation on a channel (`pop()`, `push()`,
   `net.read()`, `net.write()`) must release the GIL before blocking
   (`nb::gil_scoped_release` wrapping the call site), then re-acquire after.

Violating rule 2 deadlocks: a PyNode thread waiting to acquire the GIL cannot proceed while
another thread holds the GIL and blocks on a channel waiting for that PyNode to produce.

---

## Error Handling Contract

| Situation | Behaviour |
|---|---|
| FIFO overflow | `ChannelOverflowError` thrown in producer thread → `ErrorHandler` |
| Node function throws | Exception pointer captured → `ErrorHandler` |
| Type mismatch (C++) | `static_assert` at `connect()` compile time |
| Type mismatch (Python) | `kpn.TypeError` raised at `net.connect()` call |
| Cycle in graph | `NetworkCycleError` thrown at `build()` time |
| Thread fails to stop | Watchdog warning after configurable timeout |
| `from_python` / `to_python` fails | Exception propagated to `ErrorHandler` |

---

## Future Extension Points (Heterogeneous Execution)

Not implemented now, but the design must not close these doors:

- **`IChannel` abstract interface** — `Channel<T>` and a future `RemoteChannel<T>` (wrapping
  a socket/queue) would share the same `push`/`pop` interface. Nodes never know whether
  their channel is in-process or remote.

- **`Serializer<T>` trait** — parallel to `PythonConverter<T>` and `channel_storage_policy`,
  a specialisable trait for cross-device serialisation (MessagePack for ESP32, pinned memory
  for GPU zero-copy, etc.).

- **`NodeKind` tag** — `enum class NodeKind { Local, Gpu, Remote }` on the `INode`
  interface, letting the watchdog apply different health-check and timeout strategies per
  device type.

These three extension points are sufficient to support GPU and embedded/network targets
without redesigning the core.

---

## Thread Model

**v1: one `std::thread` per node.** This maps directly to KPN theory and is simple to reason
about. It does not scale to networks with hundreds of nodes but is appropriate for the
typical use case (tens of nodes, each doing non-trivial work).

`std::jthread` (C++20) is preferred over `std::thread` where available, as it provides a
built-in `stop_token` that simplifies the `stop()` / `try_pop` shutdown pattern.

A future executor/thread-pool model (where multiple nodes share a pool of threads and are
scheduled cooperatively) is a possible v2 extension. The `INode` interface is designed to
not assume a 1:1 thread mapping.

---

## Platform and Compiler Requirements

C++20 is required. Specific features used:

| Feature | Header / Standard | Min compiler |
|---|---|---|
| NTTP structural types (`fixed_string`) | language | GCC 11, Clang 13, MSVC 19.29 |
| `std::is_trivially_copyable_v` | `<type_traits>` | C++17+ |
| `std::jthread` + `stop_token` | `<thread>` | GCC 11, Clang 14, MSVC 19.29 |
| `if constexpr`, fold expressions | language | C++17+ |
| `auto` NTTPs | language | C++20 |
| Concepts (`requires`) | language | GCC 10, Clang 10 |

**Minimum supported compilers:** GCC 11, Clang 13, MSVC 19.29 (VS 2022).
nanobind requires Python 3.8+ and a C++17-capable compiler (satisfied by the above).

---

## Testing Strategy

Test frameworks: **Catch2 v3** (header-friendly, good async/threading support via
`REQUIRE_NOTHROW` + thread join patterns) and **Google Test** (for death tests and
parameterised test suites). Both are included; use Catch2 for integration/behaviour tests
and GTest for unit tests where `ASSERT_*` / `EXPECT_*` macros and death tests are
preferable.

### Hard cases to cover explicitly:

| Case | What to test |
|---|---|
| Channel blocking | `pop()` blocks until a producer pushes; unblocks exactly once per push |
| Channel overflow | `push()` beyond capacity throws `ChannelOverflowError` |
| Shutdown race | `stop()` called while a node is blocked on `pop()` — thread must exit cleanly |
| Multi-consumer | Two nodes connected to the same output channel each receive every item (fan-out) |
| Tuple unpacking | Multi-output node pushes correct type to each sub-channel |
| Cycle detection | `build()` throws `NetworkCycleError` for a graph with a cycle |
| Named port lookup | `input<"wrong">()` fires `static_assert`; `input<"right">()` resolves correctly |
| Wrong name count | `make_node` with mismatched name count fires readable `static_assert` |
| GIL deadlock | PyNode + blocking `net.read()` from Python do not deadlock |
| `from_python` failure | Exception propagates to `ErrorHandler`, network continues |
| `channel_storage_policy` | Large type is stored as `shared_ptr<const T>`; small type by value |

---

## Examples

Each example is a self-contained program under `examples/`. They are built as part of the
CMake build and serve as both documentation and smoke tests.

### `examples/01_hello_pipeline` — Basic linear pipeline

Two nodes connected in sequence. Demonstrates `make_node`, `Network` builder,
index-based port connection, `start_all` / `stop_all`.

```cpp
// producer → transform → sink
int produce()               { return 42; }
int double_it(int x)        { return x * 2; }
void print_it(int x)        { std::cout << x << '\n'; }

auto src  = make_node<produce>(5);
auto dbl  = make_node<double_it>(5);
auto sink = make_node<print_it>(5);

Network net;
net.add("src",  src)
   .add("dbl",  dbl)
   .add("sink", sink)
   .connect("src",  src.output<0>(),  "dbl",  dbl.input<0>())
   .connect("dbl",  dbl.output<0>(),  "sink", sink.input<0>())
   .build();
net.start_all();
```

### `examples/02_named_ports` — Named port access

Same pipeline but using `in<>` / `out<>` name tags and named port access. Demonstrates
`fixed_string` NTTP dispatch and the `static_assert` on wrong names.

```cpp
auto dbl = make_node<double_it, in<"value">, out<"result">>(5);
// ...
.connect("src", src.output<0>(), "dbl", dbl.input<"value">())
.connect("dbl", dbl.output<"result">(), "sink", sink.input<0>())
```

### `examples/03_multi_output` — Tuple-returning node / sub-port routing

A single node returns `std::tuple<Image, Mask>`. Each output is routed to a different
downstream node. Demonstrates tuple normalisation, per-element channel push, and
`output<1>()` sub-indexing.

```cpp
std::tuple<Image, Mask> detect(Image in) { ... }
void show_image(const Image& img)        { ... }
void save_mask(const Mask& m)            { ... }

// detect:output<0> → show_image, detect:output<1> → save_mask
```

### `examples/04_storage_policy` — `channel_storage_policy` specialisation

Shows the default behaviour (large struct stored as `shared_ptr<const T>`, small int by
value) and a user specialisation that overrides the default for a custom type.

```cpp
struct BigFrame { uint8_t pixels[1920*1080*3]; };
// stored as shared_ptr<const BigFrame> automatically

struct Tiny { float x, y; };  // 8 bytes — by value by default
template<> struct channel_storage_policy<Tiny> { static constexpr bool by_value = true; };
```

### `examples/05_error_handling` — Overflow and node exceptions

Demonstrates `ChannelOverflowError` (producer faster than consumer, tiny FIFO), custom
`ErrorHandler`, and a node that throws mid-execution.

```cpp
net.set_error_handler([](std::string_view name, std::exception_ptr ep) {
    try { std::rethrow_exception(ep); }
    catch (const std::exception& e) {
        std::cerr << "[" << name << "] " << e.what() << '\n';
    }
});
```

### `examples/06_watchdog` — Orchestrator / watchdog

A node that artificially stalls. Shows watchdog warning emission, configurable interval,
and graceful shutdown after a timeout.

```cpp
net.set_watchdog_interval(std::chrono::milliseconds(200));
// stall_node sleeps for 2s per item — watchdog fires warning after 200ms
```

### `examples/07_python_network` — PyNetwork with C++ and Python nodes

Python script that imports `kpn`, registers C++ node types via `make_py_network`, adds a
pure Python processing node, connects them, and runs the graph.

```python
import kpn

net = kpn.make_network([kpn.BlurNode, kpn.DetectNode])

def py_filter(img):
    return img[::2, ::2]   # downsample in Python

net.add_node("blur",      kpn.BlurNode,   inputs=["img"])
net.add_node("downsample",py_filter,      inputs=["img"], outputs=["img"])
net.add_node("detect",    kpn.DetectNode, inputs=["img"])
net.connect("blur", 0, "downsample", 0)
net.connect("downsample", 0, "detect", 0)
net.start()
```

### `examples/09_opencv_cellshade` — Real-time cell-shading with OpenCV (optional)

Captures live video from a system camera and applies a cell-shading effect entirely inside
a KPN++ graph. Built only when OpenCV is found at CMake time; skipped silently otherwise.

**Graph topology:**

```
[capture] ──Mat──> [split] ──B──> [median_b] ──B──┐
                         ├──G──> [median_g] ──G──┤
                         └──R──> [median_r] ──R──┴──> [merge] ──Mat──> [combine] ──> [display]
[capture] ──Mat──────────────────────> [detect_edges] ──mask──────────> [combine]
```

Effect steps:
1. **`split_channels`** — `cv::split` into three single-channel `cv::Mat` planes.
2. **`median_b/g/r`** — independent `cv::medianBlur(kernel=15)` per channel; large kernel
   posterises colours into flat cartoon-like regions and runs in parallel across channels.
3. **`merge_channels`** — `cv::merge` back to BGR.
4. **`detect_edges`** — greyscale, `cv::Canny`, then `cv::dilate` to produce thick outlines.
5. **`combine`** — zeros out BGR pixels wherever the edge mask is non-zero → black outlines
   drawn over the flat-colour image.
6. **`display`** — `cv::imshow`; ESC key signals shutdown via `g_running` atomic.

Demonstrates: named ports, fan-out from a single node to two downstream paths, parallel
per-channel processing, multi-input `combine` node, and error handler driving graceful stop.

```cpp
// Build only if OpenCV is present:
// cmake .. -DKPN_BUILD_EXAMPLES=ON
// ./09_opencv_cellshade [camera_index]   # default: 0
```

### `examples/08_python_subport` — Python sub-value tap and inject

Shows `net.read("node", output=N)` and `net.write("node", input=N, value=v)` from Python,
plus connecting a C++ tuple output sub-port directly to a Python node input.

```python
# Tap only output<1> (Mask) of a C++ detect node into Python
net.connect("detect", 1, "py_thresh", 0)
val = net.read("detect", output=0)   # blocks until Image is available
net.write("blur", input=1, value=1.5)  # inject sigma
```

---

## Component 8 — Web Debug UI (optional, compile-time toggle)

An optional in-process HTTP server that serves a live graph visualisation of the running
network. Zero cost when disabled — no symbols compiled in, no headers pulled.

### Toggle

```cpp
// Before any kpn include — enables the web debug server
#define KPN_WEB_DEBUG 1
#include <kpn/kpn.hpp>
```

CMake projects that want it globally:

```cmake
option(KPN_WEB_DEBUG "Enable KPN++ web debug UI" OFF)
if(KPN_WEB_DEBUG)
    target_compile_definitions(my_app PRIVATE KPN_WEB_DEBUG=1)
    # cpp-httplib is fetched automatically by CMake when this flag is ON
endif()
```

### Implementation

`include/kpn/web_debug.hpp` — included by `network.hpp` only when `KPN_WEB_DEBUG` is defined.

Depends on **cpp-httplib** (single-header, no external process, no Python required).
Served on `localhost:9090` by default (configurable via `net.set_web_debug_port(uint16_t)`).

When enabled, `Network` gains:

```cpp
#ifdef KPN_WEB_DEBUG
void set_web_debug_port(uint16_t port);   // default 9090
void start_web_debug();                   // called internally by start()
void stop_web_debug();                    // called internally by stop()
#endif
```

`start()` automatically calls `start_web_debug()` when `KPN_WEB_DEBUG` is defined.

### Endpoints

| Endpoint | Method | Description |
|---|---|---|
| `/` | GET | Serves the single-page HTML app (inline, no files needed) |
| `/api/snapshot` | GET | Returns a JSON snapshot of all node and channel stats |

The HTML page is embedded as a C++ string literal — no asset files to deploy.

### JSON Snapshot Format

```json
{
  "nodes": [
    { "id": "src",  "frames": 120, "ema_exec_ms": 33.2, "max_exec_ms": 45.1,
      "blocked_ms": 0.1, "fps": 29.8 },
    { "id": "quant", "frames": 120, "ema_exec_ms": 4.1, ... }
  ],
  "edges": [
    { "source": "src", "target": "quant", "label": "colour",
      "fill_pct": 12.5, "peak_pct": 87.5, "capacity": 8, "current": 1,
      "pushes": 120, "drops": 0, "overflows": 0 }
  ]
}
```

Node `id` comes from the name registered via `net.add("name", node)`.
Edge `label` comes from the channel name registered via `connect()` (format: `"src:N → dst:M"`).

### Browser UI

The page polls `/api/snapshot` every **500 ms** and renders a **D3.js v7 force-directed
graph**:

- **Nodes** — circles labelled with node name; colour encodes exec load:
  - green (`ema_exec_ms` < 10ms), yellow (10–50ms), orange (50–100ms), red (>100ms)
  - hover tooltip shows: frames, ema_exec_ms, max_exec_ms, blocked_ms, fps
- **Edges** — directed arrows labelled with the channel name and fill%; colour:
  - green (fill < 50%), yellow (50–80%), red (≥80%) — matches the `<<<` flag in the text report
  - hover tooltip shows: pushes, drops, overflows, capacity

D3 is loaded from CDN (`d3js.org`). The entire UI is a single inline HTML string in
`web_debug.hpp` — no file serving, no build step for assets.

### Thread model

`start_web_debug()` launches a `std::jthread` running `httplib::Server::listen()`.
The server is stopped via `httplib::Server::stop()` called from `stop_web_debug()`.
`/api/snapshot` calls `collect_snapshots()` (already thread-safe — reads atomics with
relaxed ordering) and serialises to JSON using a minimal hand-rolled serialiser
(no third-party JSON library required).

### Example usage

```cpp
#define KPN_WEB_DEBUG 1
#include <kpn/kpn.hpp>

// ... build network as normal ...
net.set_web_debug_port(9090);  // optional, 9090 is the default
net.start();
// Open http://localhost:9090 in a browser
```

---

## CMake Layout

| Target | Type | Notes |
|---|---|---|
| `kpn` | header-only interface library | C++20, no external deps |
| `kpn_python` | nanobind shared library | links `kpn`, requires Python 3.8+ |
| `kpn_tests` | executable | Catch2 v3 + Google Test |
| `kpn_examples` | executables (one per example) | built by default, off with `-DKPN_EXAMPLES=OFF` |
| `kpn_web_debug` | compile-time option | `#define KPN_WEB_DEBUG 1`; fetches cpp-httplib via CMake FetchContent |

---

## Resolved Design Decisions

All major design questions are now closed:

| Question | Decision |
|---|---|
| Shutdown mechanism | `accepting_` flag per channel; `disable()` clears queue and unblocks `pop()` |
| Overflow behaviour | `ChannelOverflowError` thrown on full accepting channel; silently dropped on disabled channel |
| Network ownership | Non-owning; user declares nodes, network holds raw pointers |
| Node lifetime contract | Nodes must outlive their `Network`; declare in same scope |
| Sub-networks | `Network` implements `INode`; `expose_input`/`expose_output` define boundary ports |
| `make_py_network` | Pure C++ template; nanobind module recompilation is the registration step |
| GIL strategy | Acquire per Python callback; release while blocking on channel ops |
| Mixed-rate inputs | `latch<>` tag for ports that reuse last-seen value; blocks only on first fire; node fires at rate of `in<>` ports |