KPN/README.md

# KPN++

A C++20 Kahn Process Network (KPN) library. Each node wraps a function and runs in its own thread, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind.

---

## Requirements

| Dependency | Version | Notes |
|---|---|---|
| CMake | ≥ 3.21 | |
| C++ compiler | GCC ≥ 11, Clang ≥ 13, MSVC 19.29 | C++20 required |
| Threads | system | `find_package(Threads)` |
| nanobind | ≥ 2.1 | auto-fetched if not installed; Python ≥ 3.8 |
| Catch2 | v3 | auto-fetched for tests |
| Google Test | v1.14 | auto-fetched for tests |
| OpenCV | ≥ 4 | optional; only for example 09 |

---

## Build

```bash
cmake -B build -DKPN_BUILD_PYTHON=OFF   # core + tests + C++ examples
cmake --build build --parallel
ctest --test-dir build
```

Enable Python bindings (requires nanobind and Python dev headers):

```bash
cmake -B build -DKPN_BUILD_PYTHON=ON
cmake --build build --parallel
```

Disable examples:

```bash
cmake -B build -DKPN_BUILD_EXAMPLES=OFF
```

---

## Core Concepts

### Nodes

A node wraps any callable. Its input types are taken from the function's parameter list; its output types from the return type. Multi-output nodes return `std::tuple<...>`.

```cpp
#include <kpn/kpn.hpp>
using namespace kpn;

// Single input, single output
int double_it(int x) { return x * 2; }

// Multi-output — must return std::tuple
std::tuple<cv::Mat, cv::Mat> split(cv::Mat frame) { ... }

// Sink — void return, no output ports
void display(cv::Mat frame) { cv::imshow("out", frame); }
```

### Creating Nodes

```cpp
// No port names (index-only access)
auto node = make_node<double_it>(/*fifo_capacity=*/5);

// Named input ports only
auto node = make_node<double_it>(in<"value">{}, 5);

// Named input and output ports
auto node = make_node<double_it>(in<"value">{}, out<"result">{}, 5);

// Named output ports only (e.g. a source with no inputs)
auto node = make_node<capture>(out<"colour","grey">{}, 5);
```

Port names are NTTP `fixed_string` values — resolved entirely at compile time, zero runtime cost.

### Building a Network

`Network` is **non-owning** — declare nodes first, then register them. Nodes must outlive the network.

```cpp
auto src  = make_node<produce>(in<"x">{}, out<"value">{}, 5);
auto proc = make_node<double_it>(in<"value">{}, out<"result">{}, 5);

Network net;
net.add("src",  src)
   .add("proc", proc)
   .connect("src",  src.output<0>(),           "proc", proc.input<0>())   // by index
   .connect("src",  src.template output<"value">(), "proc", proc.template input<"value">())  // by name
   .build();   // runs cycle detection — throws NetworkCycleError on cycles

net.start();
// ... do work ...
net.stop();
```

> **Named port syntax in template context:** when the node variable is `auto`-deduced, use `.template output<"name">()` and `.template input<"name">()` to help the parser.

### Channel Semantics

- **Bounded FIFO**: default capacity 5, configurable per-node at construction.
- **Blocking `pop()`**: consumer blocks until data is available (KPN semantics).
- **Throwing `push()`**: throws `ChannelOverflowError` if the channel is full and accepting.
- **Silent drop on disabled channel**: after `node.stop()`, its input channels are disabled — producers that push into them have the value silently dropped. No exception, no blocking.
- **Source throttling**: source nodes (no inputs) must sleep or yield to avoid overflowing downstream FIFOs. See example 09.

### Storage Policy

Large types (`sizeof > 8` or non-trivially-copyable) are stored as `std::shared_ptr<const T>` inside the channel — no copies, shared immutable ownership. Small trivially-copyable types are stored by value.

Override the policy for a specific type:

```cpp
template<> struct kpn::channel_storage_policy<MyType> {
    static constexpr bool by_value = true;
};
```

### Shutdown

`node.stop()` / `net.stop()`:
1. Sets `accepting_ = false` on all input channels (drops in-flight pushes silently).
2. Clears any queued items from those channels.
3. Unblocks any thread blocked on `pop()` (throws `ChannelClosedError` inside `run_loop`, which exits cleanly).
4. Joins the node thread.

---

## Named Ports — Design Notes

Port names use C++20 NTTP `fixed_string`. The deduction guide is required:

```cpp
template<std::size_t N>
fixed_string(const char (&)[N]) -> fixed_string<N>;
```

`fixed_string<4>` and `fixed_string<7>` are distinct types — `input<"img">()` and `input<"sigma">()` resolve to different template instantiations at compile time. Wrong names produce a `static_assert` at the call site with a readable message.

---

## Sub-Networks

`Network` implements `INode`, so it can be nested inside a larger `Network`:

```cpp
// Inner sub-network
Network pipe;
pipe.add("pre", pre_node)
    .add("enh", enh_node)
    .connect("pre", pre_node.output<0>(), "enh", enh_node.input<0>())
    .expose_input("img",    pre_node.input<0>())
    .expose_output("result", enh_node.output<0>())
    .build();

// Outer network
Network top;
top.add("pipe", pipe)
   .add("sink", sink_node)
   .connect("pipe", pipe.output<"result">(), "sink", sink_node.input<0>())
   .build();
top.start();
```

---

## Display / GUI Nodes

**Do not wrap `imshow`/`waitKey` as a KPN node.** Qt and Wayland require these to run on the main thread (the thread that owns the event loop). Instead, wire the final output channel to a `Channel<cv::Mat>` and pop it on the main thread:

```cpp
Channel<cv::Mat> result_ch(8);
comp.set_output_channel<0>(&result_ch);
// ... build and start network ...

// Main thread display loop
while (true) {
    cv::Mat frame;
    if (!result_ch.try_pop(frame, std::chrono::milliseconds(100))) continue;
    cv::imshow("output", frame);
    if (cv::waitKey(1) == 'q') break;
}
net.stop();
```

---

## Python Bindings

> Python bindings are scaffolded but not yet fully implemented. See `python/kpn_python.cpp` and `include/kpn/python/bindings.hpp`.

A `PyNetwork` is constructed from a closed list of C++ node types. The variant of all port types is derived at compile time — no runtime type registration needed.

**GIL rules (non-negotiable):**
- Acquire the GIL only for the duration of a Python callable invocation.
- Release the GIL before any blocking channel operation (`pop()`, `push()`, `net.read()`, `net.write()`).

Violating the second rule deadlocks.

---

## Examples

| Example | What it shows |
|---|---|
| `01_hello_pipeline` | Linear pipeline, index-based port wiring |
| `02_named_ports` | `in<>`/`out<>` name tags, named port access |
| `03_multi_output` | Tuple-returning node, per-element sub-port routing |
| `04_storage_policy` | `channel_storage_policy` default and specialisation |
| `05_error_handling` | `ChannelOverflowError`, `ErrorHandler` |
| `06_watchdog` | Watchdog interval, stall detection |
| `07_python_network` | PyNetwork, pure Python node *(pending)* |
| `08_python_subport` | `net.read`, `net.write`, sub-port tap *(pending)* |
| `09_opencv_cellshade` | Real-time cell-shading on webcam/pattern; requires OpenCV ≥ 4 |

Run the cell-shading example:

```bash
./build/examples/09_opencv_cellshade
# Press 'q' or close the window to stop.
# Falls back to an animated synthetic pattern if no webcam is found.
```

---

## Project Structure

```
include/kpn/
  fixed_string.hpp   — NTTP string, in<>/out<> tags, index_of
  traits.hpp         — function_traits, normalised_return_t, output_count_v
  channel.hpp        — Channel<T>, channel_storage_policy, exceptions
  port.hpp           — InputPort<N,I>, OutputPort<N,I>
  node.hpp           — Node<Func,in<...>,out<...>>, make_node, INode
  network.hpp        — Network (builder, cycle detection, watchdog)
  variant_node.hpp   — VariantNode, PythonConverter<T>, unique_types (Python layer)
  python/
    bindings.hpp     — nanobind helpers, GIL rule documentation
  kpn.hpp            — umbrella header
src/
  network.cpp        — non-template Network implementation
tests/
  test_fixed_string.cpp
  test_traits.cpp
  test_channel.cpp
  test_node.cpp
  test_network.cpp
python/
  kpn_python.cpp     — nanobind module entry point
examples/
  01_hello_pipeline/ … 09_opencv_cellshade/
```