Compare commits

..

No commits in common. "66feb918217e6c3046fd4975019da259a8bf69f3" and "949c8134efb38030b47387044f5929e64745be39" have entirely different histories.

10 changed files with 34 additions and 157 deletions

View File

@ -63,8 +63,7 @@ jobs:
if: ${{ needs.changes.outputs.dockerfile == 'true' }} if: ${{ needs.changes.outputs.dockerfile == 'true' }}
uses: ./.gitea/workflows/docker.yaml uses: ./.gitea/workflows/docker.yaml
with: with:
# Explicit string, not a boolean expression (act_runner mangles bools). push: ${{ github.event_name != 'pull_request' }}
push: ${{ github.event_name == 'pull_request' && 'false' || 'true' }}
# Runs after docker (if docker ran). A skipped docker job is fine; a failed # Runs after docker (if docker ran). A skipped docker job is fine; a failed
# one blocks this via !failure(). Re-run tests when code OR the image changed. # one blocks this via !failure(). Re-run tests when code OR the image changed.

View File

@ -4,22 +4,19 @@ name: '🐳 Builder Image'
# It is called by ci.yaml only when Dockerfile.builder or docs/requirements.txt # It is called by ci.yaml only when Dockerfile.builder or docs/requirements.txt
# change. It runs on the host runner (NOT inside the builder container) because # change. It runs on the host runner (NOT inside the builder container) because
# it needs the Docker CLI/daemon. # it needs the Docker CLI/daemon.
# Note: `push` is a STRING ("true"/"false"), not a boolean. Gitea's act_runner
# mangles boolean inputs passed from an expression (they arrive as false), so we
# pass an explicit string and compare with == 'true' below.
on: on:
workflow_call: workflow_call:
inputs: inputs:
push: push:
description: 'Push the built image to the registry ("true"/"false")' description: 'Push the built image to the registry'
type: string type: boolean
default: 'true' default: true
workflow_dispatch: workflow_dispatch:
inputs: inputs:
push: push:
description: 'Push the built image to the registry ("true"/"false")' description: 'Push the built image to the registry'
type: string type: boolean
default: 'true' default: true
jobs: jobs:
build: build:
@ -51,7 +48,7 @@ jobs:
. .
- name: Push builder image - name: Push builder image
if: ${{ inputs.push == 'true' }} if: ${{ inputs.push }}
run: | run: |
docker push gitea.tourolle.paris/dtourolle/kpnpp-builder:latest docker push gitea.tourolle.paris/dtourolle/kpnpp-builder:latest
docker push gitea.tourolle.paris/dtourolle/kpnpp-builder:${{ github.sha }} docker push gitea.tourolle.paris/dtourolle/kpnpp-builder:${{ github.sha }}

View File

@ -1,4 +1,4 @@
# KPN++ Builder Image (CI: pipeline trigger v2) # KPN++ Builder Image (CI: pipeline trigger)
# Pre-built image with GCC, CMake, Ninja, and Python dev headers for building and testing KPN++ # Pre-built image with GCC, CMake, Ninja, and Python dev headers for building and testing KPN++
# Build: docker build -f Dockerfile.builder -t gitea.tourolle.paris/dtourolle/kpnpp-builder:latest . # Build: docker build -f Dockerfile.builder -t gitea.tourolle.paris/dtourolle/kpnpp-builder:latest .
# Push: docker push gitea.tourolle.paris/dtourolle/kpnpp-builder:latest # Push: docker push gitea.tourolle.paris/dtourolle/kpnpp-builder:latest

View File

@ -2,8 +2,6 @@
A C++20 Kahn Process Network (KPN) library. Each node wraps a function and runs in its own thread, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind. A C++20 Kahn Process Network (KPN) library. Each node wraps a function and runs in its own thread, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind.
📖 **[Documentation](https://pages.tourolle.paris/dtourolle/kpn/)**
--- ---
## Requirements ## Requirements
@ -400,8 +398,8 @@ Violating the second rule deadlocks.
| `04_storage_policy` | `channel_storage_policy` default and specialisation | | `04_storage_policy` | `channel_storage_policy` default and specialisation |
| `05_error_handling` | `ChannelOverflowError`, `ErrorHandler` | | `05_error_handling` | `ChannelOverflowError`, `ErrorHandler` |
| `06_watchdog` | Watchdog interval, stall detection | | `06_watchdog` | Watchdog interval, stall detection |
| `07_python_network` | PyNetwork with a pure-Python node between a C++ source and sink | | `07_python_network` | PyNetwork, pure Python node *(pending)* |
| `08_python_subport` | Drive a Python node from Python via `net.write`/`net.read` sub-port taps | | `08_python_subport` | `net.read`, `net.write`, sub-port tap *(pending)* |
| `09_opencv_cellshade` | Real-time cell-shading on webcam/pattern; requires OpenCV ≥ 4 | | `09_opencv_cellshade` | Real-time cell-shading on webcam/pattern; requires OpenCV ≥ 4 |
Run the cell-shading example: Run the cell-shading example:

View File

@ -2,8 +2,6 @@
A C++20 Kahn Process Network (KPN) library. Each node wraps a function and runs in its own thread, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind. A C++20 Kahn Process Network (KPN) library. Each node wraps a function and runs in its own thread, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind.
📖 **[Documentation](https://pages.tourolle.paris/dtourolle/kpn/)**
--- ---
## Requirements ## Requirements
@ -222,8 +220,8 @@ Violating the second rule deadlocks.
| `04_storage_policy` | `channel_storage_policy` default and specialisation | | `04_storage_policy` | `channel_storage_policy` default and specialisation |
| `05_error_handling` | `ChannelOverflowError`, `ErrorHandler` | | `05_error_handling` | `ChannelOverflowError`, `ErrorHandler` |
| `06_watchdog` | Watchdog interval, stall detection | | `06_watchdog` | Watchdog interval, stall detection |
| `07_python_network` | PyNetwork with a pure-Python node between a C++ source and sink | | `07_python_network` | PyNetwork, pure Python node *(pending)* |
| `08_python_subport` | Drive a Python node from Python via `net.write`/`net.read` sub-port taps | | `08_python_subport` | `net.read`, `net.write`, sub-port tap *(pending)* |
| `09_opencv_cellshade` | Real-time cell-shading on webcam/pattern; requires OpenCV ≥ 4 | | `09_opencv_cellshade` | Real-time cell-shading on webcam/pattern; requires OpenCV ≥ 4 |
Run the cell-shading example: Run the cell-shading example:

View File

@ -30,8 +30,3 @@ net.build()
net.start() net.start()
time.sleep(0.1) time.sleep(0.1)
net.stop() net.stop()
# Drop the network deterministically: it holds the Python callable, which forms
# a reference cycle via globals(). Deleting the global breaks it so the network
# is reclaimed now rather than lingering to interpreter shutdown.
del net

View File

@ -1,55 +1,38 @@
""" """
08_python_subport drive a *Python* node from Python via write()/read() taps. 08_python_subport tap a C++ node's output from Python using net.read().
Graph: Graph:
(fed by net.write()) --int--> [py_triple] --int--> (tapped by net.read()) [ProduceNode] --int--> [DoubleItNode] --int--> (tapped by net.read())
Unlike 07, there is no C++ source or sink here: the only node in the network is The sink is Python: instead of connecting a PrintItNode, we call net.read()
a pure-Python function, py_triple. Python plays *both* the producer and the to pull values out of DoubleItNode's output directly into Python.
consumer by using the subport taps: We also demonstrate net.write() by injecting a value into DoubleItNode's input.
* net.write("py", 0, v) injects v into py_triple's input (Python -> network)
* net.read("py", 0) pulls py_triple's output back out (network -> Python)
This closes the loop the old version left as a "#todo": a value flows from
Python, through a Python node running inside the network, and back to Python.
""" """
import sys import sys
sys.path.insert(0, "build/python") # for `python examples/.../example.py` from repo root import time
import threading
sys.path.insert(0, "build/python")
import kpn_python as kpn import kpn_python as kpn
def py_triple(x: int) -> int:
return x * 3
net = kpn.Network() net = kpn.Network()
# The whole network is a single Python node with a tapped input and output. net.add("src", kpn.make_produce())
net.add_node("py", py_triple, inputs=["int"], outputs=["int"]) net.add("dbl", kpn.make_double_it())
net.connect("src", 0, "dbl", 0)
net.build() net.build()
net.start() net.start()
# Push values in from Python and read the Python node's results back out. # Collect a few values from DoubleItNode's output via Python tap
inputs = [1, 2, 7, 10, 100]
results = [] results = []
for v in inputs: for _ in range(5):
net.write("py", 0, v) # Python -> py_triple input val = net.read("dbl", 0)
results.append(net.read("py", 0)) # py_triple output -> Python results.append(val)
net.stop() net.stop()
print("inputs written from Python: ", inputs) print("values read from C++ DoubleItNode output:", results)
print("outputs read from py_triple:", results) assert all(v == 84 for v in results), f"expected all 84, got {results}"
print("all correct (42 * 2 = 84)")
expected = [v * 3 for v in inputs]
assert results == expected, f"expected {expected}, got {results}"
print("all correct (x * 3 computed by a Python node inside the network)")
# Drop the network deterministically. The network holds the Python callable,
# which (via globals) forms a reference cycle; deleting the global breaks it so
# the network is reclaimed promptly instead of lingering to interpreter exit.
del net

View File

@ -13,25 +13,6 @@ function(kpn_example name)
) )
endfunction() endfunction()
# Register a Python example script as a CTest smoke test. Runs the script with
# PYTHONPATH pointing at the freshly-built kpn_python module, so it does not
# depend on the caller's working directory or a hard-coded "build/python" path.
function(kpn_python_example name)
if(NOT KPN_BUILD_PYTHON)
return()
endif()
add_test(
NAME example_${name}
COMMAND ${CMAKE_COMMAND} -E env
"PYTHONPATH=$<TARGET_FILE_DIR:kpn_python>"
${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/${name}/example.py
)
set_tests_properties(example_${name} PROPERTIES
TIMEOUT 15
LABELS examples
)
endfunction()
kpn_example(01_hello_pipeline) kpn_example(01_hello_pipeline)
kpn_example(02_named_ports) kpn_example(02_named_ports)
kpn_example(03_multi_output) kpn_example(03_multi_output)
@ -50,9 +31,7 @@ if(KPN_WEB_DEBUG)
target_link_libraries(14_debug_hub PRIVATE kpn) target_link_libraries(14_debug_hub PRIVATE kpn)
kpn_target_enable_web_debug(14_debug_hub) kpn_target_enable_web_debug(14_debug_hub)
endif() endif()
# 07 and 08 are Python scripts no compiled target, but run as smoke tests. # 07 and 08 are Python scripts no compiled target needed.
kpn_python_example(07_python_network)
kpn_python_example(08_python_subport)
# 09 requires OpenCV only build if found # 09 requires OpenCV only build if found
find_package(OpenCV QUIET COMPONENTS core imgproc highgui videoio) find_package(OpenCV QUIET COMPONENTS core imgproc highgui videoio)

View File

@ -247,7 +247,7 @@ void bind_network(nb::module_& m) {
nb::class_<IVariantNode<Variant>>(m, "INode"); nb::class_<IVariantNode<Variant>>(m, "INode");
nb::class_<Net>(m, "Network", nb::type_slots(network_type_slots<Variant>())) nb::class_<Net>(m, "Network")
.def("__init__", [](Net* self) { .def("__init__", [](Net* self) {
new (self) Net(); new (self) Net();
register_all_converters<Registry>(*self); register_all_converters<Registry>(*self);

View File

@ -35,16 +35,6 @@ public:
using VNode = IVariantNode<Variant>; using VNode = IVariantNode<Variant>;
using VChannel = IVariantChannel<Variant>; using VChannel = IVariantChannel<Variant>;
// ── GC support ────────────────────────────────────────────────────────────
// Visit every Python object this network transitively holds (currently the
// callable of each PyNode). Used by the Network type's tp_traverse slot so
// Python's cyclic GC can discover instance → callable → globals() cycles.
// Defined out-of-line below, once PyNode is a complete type.
template<typename Fn>
void visit_python_objects(Fn&& visit) const;
// Drop all Python references held by nodes, breaking any cycle (tp_clear).
void clear_python_objects();
// ── Builder API ─────────────────────────────────────────────────────────── // ── Builder API ───────────────────────────────────────────────────────────
void add(std::string name, std::shared_ptr<VNode> node) { void add(std::string name, std::shared_ptr<VNode> node) {
@ -377,14 +367,6 @@ public:
out_channels_[i] = std::move(ch); out_channels_[i] = std::move(ch);
} }
// ── GC support (tp_traverse / tp_clear on the owning Network) ──────────────
// The node holds a Python callable, which typically forms an
// instance → callable → globals() → instance cycle. Expose the callable so
// the Network's GC slots can traverse and clear it. See bindings.hpp's
// network_tp_traverse/network_tp_clear.
const nb::object& python_callable() const { return callable_; }
void clear_python_callable() { callable_ = nb::object(); }
private: private:
void run_loop() { void run_loop() {
while (!stop_flag_.load(std::memory_order_relaxed)) { while (!stop_flag_.load(std::memory_order_relaxed)) {
@ -457,60 +439,6 @@ private:
NodeStats stats_; NodeStats stats_;
}; };
// ── PyNetwork GC helpers (defined here: PyNode is now complete) ────────────────
template<typename Variant>
template<typename Fn>
void PyNetwork<Variant>::visit_python_objects(Fn&& visit) const {
for (const auto& [name, node] : nodes_)
if (auto* py = dynamic_cast<const PyNode<Variant>*>(node.get()))
visit(py->python_callable());
}
template<typename Variant>
void PyNetwork<Variant>::clear_python_objects() {
for (auto& [name, node] : nodes_)
if (auto* py = dynamic_cast<PyNode<Variant>*>(node.get()))
py->clear_python_callable();
}
// ── GC type slots for the Network binding ─────────────────────────────────────
// The Network holds Python callables (via PyNode), forming uncollectable
// instance → callable → globals() → instance cycles at interpreter shutdown.
// These slots let Python's cyclic collector traverse and break them, silencing
// nanobind's leak warnings. See the nanobind "Reference leaks" documentation.
template<typename Variant>
int network_tp_traverse(PyObject* self, visitproc visit, void* arg) {
Py_VISIT(Py_TYPE(self));
if (!nb::inst_ready(self))
return 0;
auto* net = nb::inst_ptr<PyNetwork<Variant>>(self);
int rv = 0;
net->visit_python_objects([&](const nb::object& obj) {
if (rv == 0 && obj.is_valid())
rv = visit(obj.ptr(), arg);
});
return rv;
}
template<typename Variant>
int network_tp_clear(PyObject* self) {
auto* net = nb::inst_ptr<PyNetwork<Variant>>(self);
net->clear_python_objects();
return 0;
}
template<typename Variant>
PyType_Slot* network_type_slots() {
static PyType_Slot slots[] = {
{ Py_tp_traverse, reinterpret_cast<void*>(&network_tp_traverse<Variant>) },
{ Py_tp_clear, reinterpret_cast<void*>(&network_tp_clear<Variant>) },
{ 0, nullptr }
};
return slots;
}
// ── register_py_network (legacy helper) ─────────────────────────────────────── // ── register_py_network (legacy helper) ───────────────────────────────────────
// Registers PyNetwork<Variant> with the given nanobind module. // Registers PyNetwork<Variant> with the given nanobind module.
// Prefer bind_network<Registry> from auto_bind.hpp for new code. // Prefer bind_network<Registry> from auto_bind.hpp for new code.
@ -519,7 +447,7 @@ template<typename Variant>
void register_py_network(nb::module_& m, const char* class_name = "Network") { void register_py_network(nb::module_& m, const char* class_name = "Network") {
using Net = PyNetwork<Variant>; using Net = PyNetwork<Variant>;
nb::class_<Net>(m, class_name, nb::type_slots(network_type_slots<Variant>())) nb::class_<Net>(m, class_name)
.def(nb::init<>()) .def(nb::init<>())
.def("connect", &Net::connect, .def("connect", &Net::connect,
nb::arg("src"), nb::arg("out_idx"), nb::arg("src"), nb::arg("out_idx"),