First attempt
This commit is contained in:
@@ -0,0 +1,257 @@
|
||||
# KPN++
|
||||
|
||||
A C++20 Kahn Process Network (KPN) library. Each node wraps a function and runs in its own thread, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
| Dependency | Version | Notes |
|
||||
|---|---|---|
|
||||
| CMake | ≥ 3.21 | |
|
||||
| C++ compiler | GCC ≥ 11, Clang ≥ 13, MSVC 19.29 | C++20 required |
|
||||
| Threads | system | `find_package(Threads)` |
|
||||
| nanobind | ≥ 2.1 | auto-fetched if not installed; Python ≥ 3.8 |
|
||||
| Catch2 | v3 | auto-fetched for tests |
|
||||
| Google Test | v1.14 | auto-fetched for tests |
|
||||
| OpenCV | ≥ 4 | optional; only for example 09 |
|
||||
|
||||
---
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
cmake -B build -DKPN_BUILD_PYTHON=OFF # core + tests + C++ examples
|
||||
cmake --build build --parallel
|
||||
ctest --test-dir build
|
||||
```
|
||||
|
||||
Enable Python bindings (requires nanobind and Python dev headers):
|
||||
|
||||
```bash
|
||||
cmake -B build -DKPN_BUILD_PYTHON=ON
|
||||
cmake --build build --parallel
|
||||
```
|
||||
|
||||
Disable examples:
|
||||
|
||||
```bash
|
||||
cmake -B build -DKPN_BUILD_EXAMPLES=OFF
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Nodes
|
||||
|
||||
A node wraps any callable. Its input types are taken from the function's parameter list; its output types from the return type. Multi-output nodes return `std::tuple<...>`.
|
||||
|
||||
```cpp
|
||||
#include <kpn/kpn.hpp>
|
||||
using namespace kpn;
|
||||
|
||||
// Single input, single output
|
||||
int double_it(int x) { return x * 2; }
|
||||
|
||||
// Multi-output — must return std::tuple
|
||||
std::tuple<cv::Mat, cv::Mat> split(cv::Mat frame) { ... }
|
||||
|
||||
// Sink — void return, no output ports
|
||||
void display(cv::Mat frame) { cv::imshow("out", frame); }
|
||||
```
|
||||
|
||||
### Creating Nodes
|
||||
|
||||
```cpp
|
||||
// No port names (index-only access)
|
||||
auto node = make_node<double_it>(/*fifo_capacity=*/5);
|
||||
|
||||
// Named input ports only
|
||||
auto node = make_node<double_it>(in<"value">{}, 5);
|
||||
|
||||
// Named input and output ports
|
||||
auto node = make_node<double_it>(in<"value">{}, out<"result">{}, 5);
|
||||
|
||||
// Named output ports only (e.g. a source with no inputs)
|
||||
auto node = make_node<capture>(out<"colour","grey">{}, 5);
|
||||
```
|
||||
|
||||
Port names are NTTP `fixed_string` values — resolved entirely at compile time, zero runtime cost.
|
||||
|
||||
### Building a Network
|
||||
|
||||
`Network` is **non-owning** — declare nodes first, then register them. Nodes must outlive the network.
|
||||
|
||||
```cpp
|
||||
auto src = make_node<produce>(in<"x">{}, out<"value">{}, 5);
|
||||
auto proc = make_node<double_it>(in<"value">{}, out<"result">{}, 5);
|
||||
|
||||
Network net;
|
||||
net.add("src", src)
|
||||
.add("proc", proc)
|
||||
.connect("src", src.output<0>(), "proc", proc.input<0>()) // by index
|
||||
.connect("src", src.template output<"value">(), "proc", proc.template input<"value">()) // by name
|
||||
.build(); // runs cycle detection — throws NetworkCycleError on cycles
|
||||
|
||||
net.start();
|
||||
// ... do work ...
|
||||
net.stop();
|
||||
```
|
||||
|
||||
> **Named port syntax in template context:** when the node variable is `auto`-deduced, use `.template output<"name">()` and `.template input<"name">()` to help the parser.
|
||||
|
||||
### Channel Semantics
|
||||
|
||||
- **Bounded FIFO**: default capacity 5, configurable per-node at construction.
|
||||
- **Blocking `pop()`**: consumer blocks until data is available (KPN semantics).
|
||||
- **Throwing `push()`**: throws `ChannelOverflowError` if the channel is full and accepting.
|
||||
- **Silent drop on disabled channel**: after `node.stop()`, its input channels are disabled — producers that push into them have the value silently dropped. No exception, no blocking.
|
||||
- **Source throttling**: source nodes (no inputs) must sleep or yield to avoid overflowing downstream FIFOs. See example 09.
|
||||
|
||||
### Storage Policy
|
||||
|
||||
Large types (`sizeof > 8` or non-trivially-copyable) are stored as `std::shared_ptr<const T>` inside the channel — no copies, shared immutable ownership. Small trivially-copyable types are stored by value.
|
||||
|
||||
Override the policy for a specific type:
|
||||
|
||||
```cpp
|
||||
template<> struct kpn::channel_storage_policy<MyType> {
|
||||
static constexpr bool by_value = true;
|
||||
};
|
||||
```
|
||||
|
||||
### Shutdown
|
||||
|
||||
`node.stop()` / `net.stop()`:
|
||||
1. Sets `accepting_ = false` on all input channels (drops in-flight pushes silently).
|
||||
2. Clears any queued items from those channels.
|
||||
3. Unblocks any thread blocked on `pop()` (throws `ChannelClosedError` inside `run_loop`, which exits cleanly).
|
||||
4. Joins the node thread.
|
||||
|
||||
---
|
||||
|
||||
## Named Ports — Design Notes
|
||||
|
||||
Port names use C++20 NTTP `fixed_string`. The deduction guide is required:
|
||||
|
||||
```cpp
|
||||
template<std::size_t N>
|
||||
fixed_string(const char (&)[N]) -> fixed_string<N>;
|
||||
```
|
||||
|
||||
`fixed_string<4>` and `fixed_string<7>` are distinct types — `input<"img">()` and `input<"sigma">()` resolve to different template instantiations at compile time. Wrong names produce a `static_assert` at the call site with a readable message.
|
||||
|
||||
---
|
||||
|
||||
## Sub-Networks
|
||||
|
||||
`Network` implements `INode`, so it can be nested inside a larger `Network`:
|
||||
|
||||
```cpp
|
||||
// Inner sub-network
|
||||
Network pipe;
|
||||
pipe.add("pre", pre_node)
|
||||
.add("enh", enh_node)
|
||||
.connect("pre", pre_node.output<0>(), "enh", enh_node.input<0>())
|
||||
.expose_input("img", pre_node.input<0>())
|
||||
.expose_output("result", enh_node.output<0>())
|
||||
.build();
|
||||
|
||||
// Outer network
|
||||
Network top;
|
||||
top.add("pipe", pipe)
|
||||
.add("sink", sink_node)
|
||||
.connect("pipe", pipe.output<"result">(), "sink", sink_node.input<0>())
|
||||
.build();
|
||||
top.start();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Display / GUI Nodes
|
||||
|
||||
**Do not wrap `imshow`/`waitKey` as a KPN node.** Qt and Wayland require these to run on the main thread (the thread that owns the event loop). Instead, wire the final output channel to a `Channel<cv::Mat>` and pop it on the main thread:
|
||||
|
||||
```cpp
|
||||
Channel<cv::Mat> result_ch(8);
|
||||
comp.set_output_channel<0>(&result_ch);
|
||||
// ... build and start network ...
|
||||
|
||||
// Main thread display loop
|
||||
while (true) {
|
||||
cv::Mat frame;
|
||||
if (!result_ch.try_pop(frame, std::chrono::milliseconds(100))) continue;
|
||||
cv::imshow("output", frame);
|
||||
if (cv::waitKey(1) == 'q') break;
|
||||
}
|
||||
net.stop();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Python Bindings
|
||||
|
||||
> Python bindings are scaffolded but not yet fully implemented. See `python/kpn_python.cpp` and `include/kpn/python/bindings.hpp`.
|
||||
|
||||
A `PyNetwork` is constructed from a closed list of C++ node types. The variant of all port types is derived at compile time — no runtime type registration needed.
|
||||
|
||||
**GIL rules (non-negotiable):**
|
||||
- Acquire the GIL only for the duration of a Python callable invocation.
|
||||
- Release the GIL before any blocking channel operation (`pop()`, `push()`, `net.read()`, `net.write()`).
|
||||
|
||||
Violating the second rule deadlocks.
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
| Example | What it shows |
|
||||
|---|---|
|
||||
| `01_hello_pipeline` | Linear pipeline, index-based port wiring |
|
||||
| `02_named_ports` | `in<>`/`out<>` name tags, named port access |
|
||||
| `03_multi_output` | Tuple-returning node, per-element sub-port routing |
|
||||
| `04_storage_policy` | `channel_storage_policy` default and specialisation |
|
||||
| `05_error_handling` | `ChannelOverflowError`, `ErrorHandler` |
|
||||
| `06_watchdog` | Watchdog interval, stall detection |
|
||||
| `07_python_network` | PyNetwork, pure Python node *(pending)* |
|
||||
| `08_python_subport` | `net.read`, `net.write`, sub-port tap *(pending)* |
|
||||
| `09_opencv_cellshade` | Real-time cell-shading on webcam/pattern; requires OpenCV ≥ 4 |
|
||||
|
||||
Run the cell-shading example:
|
||||
|
||||
```bash
|
||||
./build/examples/09_opencv_cellshade
|
||||
# Press 'q' or close the window to stop.
|
||||
# Falls back to an animated synthetic pattern if no webcam is found.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
include/kpn/
|
||||
fixed_string.hpp — NTTP string, in<>/out<> tags, index_of
|
||||
traits.hpp — function_traits, normalised_return_t, output_count_v
|
||||
channel.hpp — Channel<T>, channel_storage_policy, exceptions
|
||||
port.hpp — InputPort<N,I>, OutputPort<N,I>
|
||||
node.hpp — Node<Func,in<...>,out<...>>, make_node, INode
|
||||
network.hpp — Network (builder, cycle detection, watchdog)
|
||||
variant_node.hpp — VariantNode, PythonConverter<T>, unique_types (Python layer)
|
||||
python/
|
||||
bindings.hpp — nanobind helpers, GIL rule documentation
|
||||
kpn.hpp — umbrella header
|
||||
src/
|
||||
network.cpp — non-template Network implementation
|
||||
tests/
|
||||
test_fixed_string.cpp
|
||||
test_traits.cpp
|
||||
test_channel.cpp
|
||||
test_node.cpp
|
||||
test_network.cpp
|
||||
python/
|
||||
kpn_python.cpp — nanobind module entry point
|
||||
examples/
|
||||
01_hello_pipeline/ … 09_opencv_cellshade/
|
||||
```
|
||||
Reference in New Issue
Block a user