39 Commits
Author SHA1 Message Date
dtourolle 454f72c167 chore: ignore generated ORT engine cache
🚦 CI / changes (push) Successful in 44s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Has been skipped
🚦 CI / tsan (push) Has been skipped
🚦 CI / docs (push) Has been skipped
ort_cache/ holds .ort engines built on first run from the .onnx models.
They are machine- and version-specific build products, not sources.
2026-08-04 14:04:50 +02:00
dtourolle 9c5ce5f34a fix: never drop a wake — a node must not sleep with one outstanding
🚦 CI / changes (push) Successful in 14s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Failing after 4m21s
🚦 CI / tsan (push) Failing after 3m20s
🚦 CI / docs (push) Has been skipped
28e0667 stopped nodes blocking a worker on a full output, but replaced an
intermittent hang with a quieter one: the pipeline still wedged about 2
runs in 30, now with every worker idle in pthread_cond_wait rather than
asleep in a push. Nothing was blocked; nothing had been woken.

try_submit discarded any wake arriving while queued_ was up:

    if (queued_.compare_exchange_strong(expected, true, ...))
        scheduler_->submit(...);
    // else: silently gone

Wakes are edge-triggered — a channel fires its space callback once, on the
transition — so a dropped one never returns. A node could park a value,
release its worker, and sleep forever holding exactly the output its
consumer was waiting for, while its producer parked on an input channel
that would never drain.

try_submit now records the drop in wake_pending_, and release_and_recheck()
consumes it at every site that releases a node, giving one invariant: a
node never sleeps with a wake outstanding. This subsumes the two ad-hoc
re-checks added for the parked-retry and normal push paths, which only
moved the stall (1211 items to 6472) because each new early return was a
fresh chance to drop a wake. self_stop keeps a plain store — honouring a
pending wake there would resubmit a dead node.

Adds "a saturated chain never stalls", which reproduces this in about a
second where the pipeline needed ~30 runs. It asserts *progress does not
freeze* rather than a completion total: capacity-1 channels are slow, and
slow must never be reported as wedged. Verified in both directions — it
stalls after 1211 items on 28e0667 and passes here.

The existing chain test could not catch it: 40 items drain before any
strand occurs, and its producer emits forever, so fresh input keeps
re-triggering on_input_ready() and flushing the stranded value.

Tests: 122/122.
2026-08-02 18:33:17 +02:00
dtourolle 28e06675f5 fix: park nodes on a full output instead of blocking the worker
🚦 CI / changes (push) Successful in 6s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Failing after 4m20s
🚦 CI / tsan (push) Failing after 3m16s
🚦 CI / docs (push) Has been skipped
push_blocking parked a scheduler worker inside the push. Nodes own a
private single-thread pool, so the parked thread was the only one that
could drain that node's own input — hold-and-wait, and under sustained
backpressure four nodes of a five-node chain slept in nanosleep at once.
channel.hpp already warned about this for sentinels; it applies just as
much to data pushes.

The scheduler was purely input-driven: on_input_ready() wakes a node when
input arrives, with no counterpart for "my output has room". Lacking that
signal, blocking the thread was the only way to handle a full output.
This adds the missing half.

- Channel::try_push + has_space + set_space_callback; the callback fires
  from both pop() and try_pop_now().
- PoolNode/PoolObjectNode keep a one-slot pending_ buffer with per-element
  done flags, so a retry cannot duplicate an already-accepted element. One
  slot suffices because queued_ admits at most one fire_once per node.
- The re-check after clearing queued_ closes the lost-wakeup race where a
  space callback fires while the flag is still up and is swallowed.

Two bugs surfaced once nodes actually parked, both fixed here:

- pop_one reports an *empty* channel as ChannelClosedError, which is also
  the node's "upstream finished, self-stop" signal. A node woken by output
  space with empty inputs therefore killed itself. fire_once now releases
  the worker when its inputs are not ready rather than falling through.
- The drained-park path resubmitted unconditionally instead of via
  on_input_ready(), firing nodes with nothing to read.

compute_priority is now output-aware: mean output fill is deducted from
mean input fill, mapped as 0.5·(1 + in - out). Input fill alone asks only
"how much work is waiting for me"; a node whose outputs are already full
cannot deliver, so running it just parks it again and wastes the slot
while the node that would drain that channel waits behind it. The
scheduler now favours whoever is furthest downstream of a bottleneck.

Also adds a network-level error listener. A node's exception was discarded
at the node boundary and survived only as a Closed event, which reports
that a node stopped but not why — that missing detail is what made the
above slow to diagnose. INode::set_network_error_callback plus
StaticNetwork::set_error_handler forward it to the application.

Tests: 121/121. test_backpressure_deadlock drives a five-node chain with
capacity-2 channels against a slow sink and fails on the old code. The
four test_pool_node overflow tests now assert parking rather than the
removed drop-and-report behaviour.

Known-incomplete: a rare hang remains, roughly 1 run in 20 against a 300s
timeout, down from every run failing. Committed because the fix is a large
strict improvement and the residual case needs its own reproduction.
2026-07-31 22:40:07 +02:00
dtourolleandClaude Opus 5 6595e6e925 fix: node outputs block instead of dropping on a full channel
🚦 CI / changes (push) Successful in 30s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Failing after 1h44m2s
🚦 CI / docs (push) Has been skipped
🚦 CI / tsan (push) Failing after 3h14m57s
Every node output used the throwing push(), so a consumer falling behind cost
values rather than time. push_blocking() already existed on Channel and
OutputPort — "wait for the consumer to drain instead of dropping; the producer
just runs slower" — but nothing called it.

A dropped frame does not degrade a downstream result, it silently changes one,
and the consumer has no way to tell it happened. For any pipeline whose output
is a claim about its input, that is corruption rather than degradation.

Safe because sentinels are already handled out-of-band, above this path: only
data blocks, so the EOF token that unwinds the network can always overtake a
stalled data path. That is exactly the hold-and-wait deadlock the push_sentinel
comment warns about, and the reason it is not reachable here.

Measured on a downstream consumer (face pipeline, 77s clip at 5 fps, expected
385 sampled frames):

  before  65 frames written, 320 dropped at one node, 29s
  after   385 frames written, 0 dropped, 17s

Faster, not slower — a dropped frame has already cost its decode, and the
overflow exception cost more. Two consecutive runs now produce byte-identical
output, which they did not before: what got dropped depended on timing, so the
same command could yield different results.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 11:31:53 +02:00
dtourolle 75b34f31bb Merge pull request 'feat: persistent-pipeline reuse — push_blocking, node introspection, stateful wrapper' (#2) from feature/persistent-pipeline-reuse into master
🚦 CI / changes (push) Successful in 4s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Successful in 4m39s
🚦 CI / tsan (push) Successful in 2m57s
🚦 CI / docs (push) Has been skipped
Reviewed-on: #2
2026-07-19 16:11:27 +00:00
dtourolle 4b6e498ba7 feat: persistent-pipeline reuse — push_blocking, node introspection, stateful wrapper
🚦 CI / changes (pull_request) Successful in 17s
🚦 CI / docker (pull_request) Has been skipped
🚦 CI / test (pull_request) Successful in 4m42s
🚦 CI / tsan (pull_request) Successful in 2m56s
🚦 CI / docs (pull_request) Has been skipped
Adds three pieces needed to build one KPN network and reuse it across many
replays/configs instead of tearing down and rebuilding per run:

- Channel<T>::push_blocking (+ IVariantChannel/VariantChannel forwarding):
  lossless backpressure push that waits for space instead of dropping when
  the ring is full. PyNode's run_loop now uses it so a downstream consumer
  lagging behind never silently drops a frame.
- PyNetwork::node_ptr / node_stats: raw node handle by name (for a binding
  to dynamic_cast to a concrete wrapper and call functor-specific runtime
  setters) and a per-node timing snapshot for profiling.
- ObjectVariantNodeWrapper: variant-node adapter for functors that need
  runtime-constructed state (a Config, a loaded gallery), mirroring
  VariantNodeWrapper's channel plumbing but backed by ObjectNode<Obj>.

Built and used downstream in scene-actor-extraction's sae_kpn Python replay
bindings for repeated threshold-sweep evaluation of the same pipeline.
2026-07-19 16:55:36 +02:00
dtourolle 5ecf3cde4f Merge pull request 'spec-and-tsan' (#1) from spec-and-tsan into master
🚦 CI / changes (push) Successful in 4s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Successful in 4m36s
🚦 CI / tsan (push) Successful in 2m54s
🚦 CI / docs (push) Has been skipped
Reviewed-on: #1
2026-07-17 18:11:51 +00:00
dtourolleandClaude Opus 4.8 ec19137ed9 ci: fix TSan aborting at init on the nested-LXC runner
🚦 CI / changes (pull_request) Successful in 6s
🚦 CI / docker (pull_request) Has been skipped
🚦 CI / test (pull_request) Successful in 4m32s
🚦 CI / tsan (pull_request) Successful in 2m54s
🚦 CI / docs (pull_request) Has been skipped
The ThreadSanitizer job runs on Docker nested in an unprivileged LXC
container, whose kernel randomizes mmap addresses beyond the range TSan's
fixed shadow mapping expects. TSan aborted at init with "unexpected memory
mapping" before any test ran.

Disable ASLR per-process with `setarch -R`, which needs the personality(2)
syscall that Docker's default seccomp profile blocks; seccomp=unconfined on
the container permits it. Verified on the runner that both are required:
setarch -R alone gets EPERM, seccomp alone still aborts, both together run
clean. Scoped to the tsan job, which runs only our own test binaries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 20:01:57 +02:00
dtourolleandClaude Opus 4.8 a0c4bf580e fix: deliver EOF sentinel only when the ring is freshly empty
🚦 CI / changes (pull_request) Successful in 4s
🚦 CI / docker (pull_request) Has been skipped
🚦 CI / test (pull_request) Successful in 7m30s
🚦 CI / tsan (pull_request) Failing after 2m21s
🚦 CI / docs (pull_request) Has been skipped
Channel<T>::pop() surfaced the out-of-band sentinel from its empty branch
using the tail_ snapshot taken at the top of the loop. Under contention the
producer can push more values *and* the sentinel in the window between that
snapshot and take_sentinel(), so pop() could return the sentinel while real
values still sat in the ring — the sentinel jumping ahead of values pushed
before it. No value was lost (a consumer that keeps draining still receives
them, and approx_size() keeps counting them so a PoolNode reschedules), but a
consumer treating the sentinel as a hard "last message" barrier would act on
EOF early.

Re-confirm emptiness against a fresh tail_ load before taking the sentinel.
Costs one acquire-load on the empty-ring path only; never runs in steady
state. The spin and post-spin takes already reload tail_ on the line above
them; try_pop_now() already reads tail_ fresh in the same branch — both were
correct and are unchanged.

The two sentinel stress cases now assert the strict "sentinel is last, after
every value" ordering (previously relaxed to avoid the flake this fixes).
Verified TSan-clean (2606 assertions, no data races) over repeated runs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 23:14:45 +02:00
dtourolleandClaude Opus 4.8 3ac2242df1 docs: rewrite SPEC.md to match the implemented library
The spec had drifted far from the code. Key corrections:

- Execution model is reactive (PoolNode submits fire_once() to a
  ThreadPool when inputs are ready), not one blocking thread per node
- Channel<T> is a lock-free SPSC ring buffer (atomic wait/notify +
  spin-before-sleep), not a mutex+CV queue
- Remove latch<> ports (never implemented)
- NodeErrorHandler returns bool (skip vs stop); per-node
- Document new subsystems: scheduler, InterruptNode, Router/FilterNode,
  MainThreadNode, SharedResource, DebugHub, diagnostics/stats layer
- Update StaticNetwork, Python auto_bind layer, examples 01-16

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 23:14:45 +02:00
dtourolleandClaude Opus 4.8 399ee4cf9b test: add ThreadSanitizer verification for lock-free Channel<T>
Add a contended SPSC stress suite (tests/test_channel_stress.cpp) that
actually exercises the ring's memory-ordering pairing and spin/futex/
lost-wakeup logic, plus the CMake and CI plumbing to run it under TSan:

- KPN_SANITIZER cache var + kpn_sanitizer_flags() helper (no-op when unset)
- kpn_tests_stress executable, labelled "stress" for CTest
- reusable tsan.yaml workflow (gcc:14 builder image, already ships libtsan)
- ci.yaml gains a tsan job on the same code/dockerfile triggers as test

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 23:14:45 +02:00
dtourolle 66feb91821 ci: pass docker push input as a string, not a boolean
🚦 CI / changes (push) Successful in 6s
🚦 CI / docker (push) Successful in 2m57s
🚦 CI / test (push) Successful in 8m7s
🚦 CI / docs (push) Successful in 6s
Gitea's act_runner mangles boolean workflow_call/dispatch inputs passed
from an expression -- they arrive as false regardless of value. Declare
`push` as a string ("true"/"false") and compare with == 'true' so the
builder image is pushed on non-PR events again.
2026-07-04 15:07:27 +02:00
dtourolle 903dd4eea5 docs: update README examples table and add documentation link
07_python_network and 08_python_subport now work and run as CI smoke
tests, so drop their "(pending)" markers and describe what they actually
demonstrate. Also surface the hosted documentation link at the top and
re-render README.md from README.md.in.
2026-07-04 15:07:12 +02:00
dtourolle 298c9e770b examples: run Python examples 07 and 08 as CTest smoke tests
Neither Python example was registered as a test, so `ctest -L examples`
in CI skipped them entirely -- which is how 08's missing Python node
went unnoticed.

Add a kpn_python_example() helper (gated on KPN_BUILD_PYTHON) that runs
each script with PYTHONPATH pointed at the freshly-built module, so it
does not depend on cwd or a hard-coded build/python path, and register
07 and 08. Also del the network in 07 for deterministic teardown.
2026-07-04 14:44:30 +02:00
dtourolle 2b0873b61b examples: make 08_python_subport run a real Python node
The example was named "python subport" but its graph was entirely C++
(ProduceNode -> DoubleItNode); Python only tapped the output, leaving a
dangling "#todo: return value to network".

Rewrite so the only node in the graph is a pure-Python py_triple, driven
from both ends via the subport taps: net.write() injects inputs and
net.read() pulls results back, closing the round trip. Also del the
network at the end so its callable cycle is reclaimed deterministically.
2026-07-04 14:44:24 +02:00
dtourolle c4538f03ca python: fix nanobind Network reference leak via GC type slots
The Python Network holds each PyNode's callable, forming an
uncollectable instance -> callable -> globals() -> instance cycle that
tripped nanobind's leak check at interpreter shutdown.

Implement tp_traverse/tp_clear type slots on the Network binding so
Python's cyclic collector can see through the C++-held callables and
break the cycle. PyNode exposes its callable; PyNetwork visits and
clears them. Wired into both binding sites (auto_bind and the legacy
register_py_network).
2026-07-04 14:43:47 +02:00
dtourolleandClaude Opus 4.8 949c8134ef Ignore build_test/ (out-of-tree test build dir)
🚦 CI / changes (push) Successful in 3s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Has been skipped
🚦 CI / docs (push) Has been skipped
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 11:02:54 +02:00
dtourolleandClaude Opus 4.8 19f5a2b0ae Deliver EOF sentinels out-of-band to prevent teardown deadlock
🚦 CI / changes (push) Successful in 18s
🚦 CI / docker (push) Has been skipped
🚦 CI / docs (push) Has been cancelled
🚦 CI / test (push) Has been cancelled
Channel::push() drops values on overflow (the intended backpressure
policy for data), and PoolNode swallows the resulting ChannelOverflowError.
For a control sentinel like EOF this is fatal: a single dropped EOF under
backpressure wedges every downstream pop() forever, so the pipeline never
tears down.

Deliver sentinels out-of-band instead. Channel::push_sentinel() stores the
token in a dedicated slot that does not consume ring capacity, so it can
never overflow and — crucially — never blocks the caller. That non-blocking
property is essential: each KPN node has a single worker thread, so a
*blocking* push would park that thread and stop it draining its own input,
cascading into a hold-and-wait deadlock under backpressure. The consumer's
pop()/try_pop_now() drain the ring first, then deliver the sentinel, so it
always arrives after every value pushed before it.

approx_size() (which node readiness checks call) counts a pending sentinel
as consumable work, so a channel carrying only a sentinel still schedules
its consumer's next fire — without this the token would sit undelivered and
the pipeline would still deadlock at teardown.

PoolNode/PoolObjectNode route values carrying an eof flag (direct .eof or
nested .source.eof) through push_sentinel via a SFINAE-safe is_sentinel_value
trait; all other values keep the existing lossy throwing push. The trait
compiles to false for types without an eof convention, so this is a no-op
for pipelines that don't use one.

Verified end-to-end: scene_analyze now reaches EOF, flushes its output, and
exits cleanly instead of hanging.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 00:30:40 +02:00
dtourolle a4de64ea04 Add liscence and prepare for OSS release
🚦 CI / changes (push) Successful in 39s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Has been skipped
🚦 CI / docs (push) Successful in 7s
2026-06-28 12:05:56 +02:00
dtourolleandClaude Opus 4.8 7c6a8be2b7 ci: provoke pipeline (touch Dockerfile + code)
🚦 CI / changes (push) Successful in 32s
🚦 CI / docker (push) Successful in 1m0s
🚦 CI / test (push) Successful in 4m47s
🚦 CI / docs (push) Successful in 13s
Trivial comment changes to exercise the new CI orchestration: the
docker job should rebuild+push the builder image first, then test
and docs run against the fresh image.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 09:46:35 +02:00
dtourolle 4c0f1f6923 fix CI
🚦 CI / changes (push) Successful in 33s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Has been skipped
🚦 CI / docs (push) Has been skipped
2026-06-20 09:38:26 +02:00
dtourolle 6b52526e44 Auto-build docker when docker file changes.
🚦 CI / changes (push) Failing after 5s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Has been skipped
🚦 CI / docs (push) Has been skipped
2026-06-20 09:16:27 +02:00
dtourolle 7cb92a4091 Build docs with a pre-configured docker
🧪 Test / test (push) Successful in 4m41s
2026-06-20 08:55:48 +02:00
dtourolle 20668d6955 Fix docker image
🧪 Test / test (push) Successful in 7m31s
2026-06-19 22:29:46 +02:00
dtourolle 6f384dc4b5 Added callbacks for node errors and fifo overflow
📚 Docs / deploy (push) Failing after 7s
🧪 Test / test (push) Has been cancelled
Add new doc system which should/might deploy to pages.
2026-06-19 22:26:39 +02:00
dtourolleandClaude Opus 4.8 79916f1da1 Set node names in make_network for user and fanout nodes
🧪 Test / test (push) Successful in 6m22s
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 15:20:50 +02:00
dtourolle f6bcaa15b0 Performance improvements, better readme and complete python bindings
🧪 Test / test (push) Failing after 28m30s
2026-05-12 21:23:33 +02:00
dtourolle c39db82763 Add a per node error handler possibility
🧪 Test / test (push) Successful in 6m7s
2026-05-10 19:51:23 +02:00
dtourolle 1e9ba5ee66 Add a unified observability interface for applications with multiple networks
🧪 Test / test (push) Successful in 6m9s
2026-05-10 19:08:40 +02:00
dtourolle 278c122e8f Add shared reasource tag to allow coordination of usage
🧪 Test / test (push) Successful in 6m8s
2026-05-09 15:22:27 +02:00
dtourolle 9acc42b2e9 Fix fanout naming for debug graph
🧪 Test / test (push) Failing after 2m38s
2026-05-09 09:54:40 +02:00
dtourolle da8f4d9926 Add debug graph to static network 2026-05-09 08:59:19 +02:00
dtourolle 011b5eb35f fix web debuginterface 2026-05-09 08:40:06 +02:00
dtourolle 9ce581b5ce Fixed bug when generating identical fanouts 2026-05-09 08:36:51 +02:00
dtourolle 2bca2a7554 Add static network
🧪 Test / test (push) Successful in 6m4s
2026-05-08 20:00:15 +02:00
dtourolle 3c683c821d Add more exmaples and fix CI
🧪 Test / test (push) Successful in 5m59s
2026-05-08 18:28:12 +02:00
dtourolle 127ffb3849 Fix CI
🧪 Test / test (push) Successful in 6m47s
🐳 Build Builder Image / build-and-push (push) Failing after 1s
2026-05-08 18:09:13 +02:00
dtourolle 2a5c0a0b4d Add build infra
🧪 Test / test (push) Failing after 1s
🐳 Build Builder Image / build-and-push (push) Failing after 1s
2026-05-08 18:00:03 +02:00
dtourolle 5e77dc836b First attempt 2026-05-08 17:48:16 +02:00
146 changed files with 15192 additions and 18886 deletions
+87
View File
@@ -0,0 +1,87 @@
name: '🚦 CI'
# Single orchestrator. This is the only workflow that triggers on push/PR.
# It decides which reusable sub-workflows to run and in what order:
# changes ─┬─> docker (only if the Dockerfile/requirements changed) ─┬─> test
# │ └─> docs
# When the builder image is rebuilt it MUST finish (and push) before test/docs
# run, so they validate against the fresh image.
on:
push:
branches:
- master
- develop
pull_request:
branches:
- master
- develop
workflow_dispatch:
jobs:
# Detect which parts of the repo changed in this push/PR.
changes:
runs-on: linux/amd64
# Runs in the builder image because the host has no Node, which the
# JS-based checkout/paths-filter actions require.
container:
image: gitea.tourolle.paris/dtourolle/kpnpp-builder:latest
outputs:
dockerfile: ${{ steps.filter.outputs.dockerfile }}
code: ${{ steps.filter.outputs.code }}
docs: ${{ steps.filter.outputs.docs }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Detect changed paths
id: filter
uses: dorny/paths-filter@v3
with:
filters: |
dockerfile:
- 'Dockerfile.builder'
- 'docs/requirements.txt'
docs:
- 'docs/**'
- 'mkdocs.yml'
- 'examples/**/*.cpp'
code:
- 'src/**'
- 'include/**'
- 'tests/**'
- 'examples/**'
- 'python/**'
- 'CMakeLists.txt'
- '**/*.cpp'
- '**/*.hpp'
- '**/*.h'
# Rebuild the builder image first, but only when it actually changed.
# On pull requests we build to validate the Dockerfile but do not push.
docker:
needs: changes
if: ${{ needs.changes.outputs.dockerfile == 'true' }}
uses: ./.gitea/workflows/docker.yaml
with:
# Explicit string, not a boolean expression (act_runner mangles bools).
push: ${{ github.event_name == 'pull_request' && 'false' || 'true' }}
# Runs after docker (if docker ran). A skipped docker job is fine; a failed
# one blocks this via !failure(). Re-run tests when code OR the image changed.
test:
needs: [changes, docker]
if: ${{ !failure() && !cancelled() && (needs.changes.outputs.code == 'true' || needs.changes.outputs.dockerfile == 'true') }}
uses: ./.gitea/workflows/test.yaml
# ThreadSanitizer run for the lock-free Channel<T>. Same trigger conditions as
# test (code or image changed); runs in parallel with test.
tsan:
needs: [changes, docker]
if: ${{ !failure() && !cancelled() && (needs.changes.outputs.code == 'true' || needs.changes.outputs.dockerfile == 'true') }}
uses: ./.gitea/workflows/tsan.yaml
docs:
needs: [changes, docker]
if: ${{ !failure() && !cancelled() && github.ref == 'refs/heads/master' && (needs.changes.outputs.docs == 'true' || needs.changes.outputs.dockerfile == 'true') }}
uses: ./.gitea/workflows/docs.yaml
secrets: inherit
+57
View File
@@ -0,0 +1,57 @@
name: '🐳 Builder Image'
# Reusable workflow: builds (and optionally pushes) the kpnpp-builder image.
# It is called by ci.yaml only when Dockerfile.builder or docs/requirements.txt
# change. It runs on the host runner (NOT inside the builder container) because
# it needs the Docker CLI/daemon.
# Note: `push` is a STRING ("true"/"false"), not a boolean. Gitea's act_runner
# mangles boolean inputs passed from an expression (they arrive as false), so we
# pass an explicit string and compare with == 'true' below.
on:
workflow_call:
inputs:
push:
description: 'Push the built image to the registry ("true"/"false")'
type: string
default: 'true'
workflow_dispatch:
inputs:
push:
description: 'Push the built image to the registry ("true"/"false")'
type: string
default: 'true'
jobs:
build:
runs-on: linux/amd64
steps:
# This job runs on the host (not in a container) so it can reach the
# host Docker daemon and reuse the cached registry credentials. The host
# has no Node, so the JS-based actions/checkout can't run here; do a
# minimal shallow fetch of this commit with plain git instead.
- name: Checkout repository
run: |
git init -q .
git remote add origin "${{ github.server_url }}/${{ github.repository }}.git"
git -c http.extraheader="AUTHORIZATION: basic $(printf '%s' '${{ github.actor }}:${{ github.token }}' | base64 -w0)" \
fetch --depth 1 origin "${{ github.sha }}"
git checkout -q FETCH_HEAD
# No docker login step: the host runner was authenticated to
# gitea.tourolle.paris with `docker login` during setup, so its cached
# credentials in ~/.docker/config.json cover the push below.
- name: Build builder image
# Context is the repo root because Dockerfile.builder COPYs
# docs/requirements.txt during the build.
run: |
docker build \
-f Dockerfile.builder \
-t gitea.tourolle.paris/dtourolle/kpnpp-builder:latest \
-t gitea.tourolle.paris/dtourolle/kpnpp-builder:${{ github.sha }} \
.
- name: Push builder image
if: ${{ inputs.push == 'true' }}
run: |
docker push gitea.tourolle.paris/dtourolle/kpnpp-builder:latest
docker push gitea.tourolle.paris/dtourolle/kpnpp-builder:${{ github.sha }}
+34
View File
@@ -0,0 +1,34 @@
name: '📚 Docs'
# Triggering and path filtering are owned by ci.yaml (the orchestrator), which
# calls this as a reusable workflow. workflow_dispatch is kept for manual runs.
on:
workflow_call:
workflow_dispatch:
jobs:
deploy:
runs-on: linux/amd64
container:
image: gitea.tourolle.paris/dtourolle/kpnpp-builder:latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0 # full history needed for mkdocs gh-deploy
- name: Configure git identity
run: |
git config user.name "Gitea Actions"
git config user.email "actions@gitea.tourolle.paris"
- name: Build and deploy to gitea-pages branch
env:
GH_TOKEN: ${{ github.token }}
run: |
mkdocs gh-deploy \
--force \
--remote-branch gitea-pages \
--remote-name origin \
--message "docs: deploy from ${{ github.sha }}"
+67
View File
@@ -0,0 +1,67 @@
name: '🧪 Test'
# Triggering and path filtering are owned by ci.yaml (the orchestrator), which
# calls this as a reusable workflow. workflow_dispatch is kept for manual runs.
on:
workflow_call:
workflow_dispatch:
jobs:
test:
runs-on: linux/amd64
container:
image: gitea.tourolle.paris/dtourolle/kpnpp-builder:latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
path: test-${{ github.run_id }}
- name: Cache FetchContent dependencies
uses: actions/cache@v3
with:
path: ~/.cmake/fetchcontent
key: cmake-fetchcontent-${{ hashFiles('**/CMakeLists.txt') }}
restore-keys: cmake-fetchcontent-
- name: Configure
working-directory: test-${{ github.run_id }}
run: |
cmake -S . -B build \
-G Ninja \
-DCMAKE_BUILD_TYPE=Debug \
-DKPN_BUILD_TESTS=ON \
-DKPN_BUILD_EXAMPLES=ON \
-DKPN_BUILD_PYTHON=ON \
-DFETCHCONTENT_BASE_DIR=$HOME/.cmake/fetchcontent
- name: Build
working-directory: test-${{ github.run_id }}
run: cmake --build build --parallel
- name: Run unit tests
working-directory: test-${{ github.run_id }}
run: |
cd build
ctest --output-on-failure --output-junit test-results.xml --label-exclude examples
- name: Run example smoke tests
working-directory: test-${{ github.run_id }}
run: |
cd build
ctest --output-on-failure --output-junit example-results.xml -L examples
- name: Upload test results
if: always()
uses: actions/upload-artifact@v3
with:
name: test-results
path: |
test-${{ github.run_id }}/build/test-results.xml
test-${{ github.run_id }}/build/example-results.xml
retention-days: 7
- name: Cleanup
if: always()
run: rm -rf test-${{ github.run_id }}
+77
View File
@@ -0,0 +1,77 @@
name: '🧵 ThreadSanitizer'
# Reusable workflow: builds the channel stress suite with ThreadSanitizer and
# runs it. This is the dynamic half of verifying the lock-free SPSC Channel<T>
# (the static half is the CDSChecker model-check harness in verify/).
#
# Triggering and path filtering are owned by ci.yaml (the orchestrator), which
# calls this only when code changed. workflow_dispatch is kept for manual runs.
#
# Runs in the prebuilt builder image (gcc:14), which already ships libtsan — no
# package installs at job time.
on:
workflow_call:
workflow_dispatch:
jobs:
tsan:
runs-on: linux/amd64
container:
image: gitea.tourolle.paris/dtourolle/kpnpp-builder:latest
# This runner is Docker nested in an unprivileged LXC container, whose
# kernel randomizes mmap addresses beyond the range TSan's fixed shadow
# mapping expects, so TSan aborts at init with "unexpected memory
# mapping". The fix is to disable ASLR per-process with `setarch -R`
# (below), which needs the personality(2) syscall that Docker's default
# seccomp profile blocks. seccomp=unconfined permits it. Verified on the
# runner: setarch -R alone gets EPERM, seccomp alone still aborts, both
# together run clean. Scoped to this job, which runs only our own tests.
options: --security-opt seccomp=unconfined
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
path: tsan-${{ github.run_id }}
- name: Cache FetchContent dependencies
uses: actions/cache@v3
with:
path: ~/.cmake/fetchcontent
key: cmake-fetchcontent-${{ hashFiles('**/CMakeLists.txt') }}
restore-keys: cmake-fetchcontent-
- name: Configure (TSan)
working-directory: tsan-${{ github.run_id }}
run: |
cmake -S . -B build \
-G Ninja \
-DCMAKE_BUILD_TYPE=Debug \
-DKPN_SANITIZER=thread \
-DKPN_BUILD_TESTS=ON \
-DKPN_BUILD_EXAMPLES=OFF \
-DKPN_BUILD_PYTHON=OFF \
-DFETCHCONTENT_BASE_DIR=$HOME/.cmake/fetchcontent
- name: Build (TSan)
working-directory: tsan-${{ github.run_id }}
run: cmake --build build --parallel --target kpn_tests kpn_tests_stress
- name: Run stress suite under TSan
working-directory: tsan-${{ github.run_id }}
# halt_on_error=1 makes the first detected race fail the job; the report
# (with both stacks) is printed to the log. second_deadlock_stack gives
# the full picture for lock-order issues.
env:
TSAN_OPTIONS: "halt_on_error=1 second_deadlock_stack=1"
# setarch -R disables ASLR for this process; see the container comment.
run: setarch -R ./build/tests/kpn_tests_stress
- name: Run unit tests under TSan
working-directory: tsan-${{ github.run_id }}
env:
TSAN_OPTIONS: "halt_on_error=1 second_deadlock_stack=1"
run: setarch -R ./build/tests/kpn_tests
- name: Cleanup
if: always()
run: rm -rf tsan-${{ github.run_id }}
+30
View File
@@ -0,0 +1,30 @@
# Build output
build/
build_test/
build_debug/
site/
# Python
__pycache__/
*.py[cod]
*.pyd
*.pyo
*.egg-info/
dist/
*.egg
.venv/
venv/
# Editors
.vscode/
.idea/
*.swp
*.swo
*~
# OS
.DS_Store
Thumbs.db
# Claude Code local settings
.claude/settings.local.json
include/kpn/ort_cache/
View File
-830
View File
@@ -1,830 +0,0 @@
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="A C++20 Kahn Process Network library">
<link rel="icon" href="/assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.6.1, mkdocs-material-9.7.6">
<title>KPN++</title>
<link rel="stylesheet" href="/assets/stylesheets/main.484c7ddc.min.css">
<link rel="stylesheet" href="/assets/stylesheets/palette.ab4e12ef.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<script>__md_scope=new URL("/",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="slate" data-md-color-primary="indigo" data-md-color-accent="indigo">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
</div>
<div data-md-component="announce">
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href="/." title="KPN++" class="md-header__button md-logo" aria-label="KPN++" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
KPN++
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
</span>
</div>
</div>
</div>
<form class="md-header__option" data-md-component="palette">
<input class="md-option" data-md-color-media="" data-md-color-scheme="slate" data-md-color-primary="indigo" data-md-color-accent="indigo" aria-hidden="true" type="radio" name="__palette" id="__palette_0">
</form>
<script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" tabindex="0" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://gitea.tourolle.paris/dtourolle/KPN" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
dtourolle/KPN
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<nav class="md-tabs" aria-label="Tabs" data-md-component="tabs">
<div class="md-grid">
<ul class="md-tabs__list">
<li class="md-tabs__item">
<a href="/." class="md-tabs__link">
Home
</a>
</li>
<li class="md-tabs__item">
<a href="/getting-started/" class="md-tabs__link">
Getting Started
</a>
</li>
<li class="md-tabs__item">
<a href="/nodes/" class="md-tabs__link">
Concepts
</a>
</li>
<li class="md-tabs__item">
<a href="/error-handling/" class="md-tabs__link">
Error Handling & Events
</a>
</li>
<li class="md-tabs__item">
<a href="/static-network/" class="md-tabs__link">
Advanced
</a>
</li>
<li class="md-tabs__item">
<a href="/examples/" class="md-tabs__link">
Examples
</a>
</li>
</ul>
</div>
</nav>
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary md-nav--lifted" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="/." title="KPN++" class="md-nav__button md-logo" aria-label="KPN++" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
KPN++
</label>
<div class="md-nav__source">
<a href="https://gitea.tourolle.paris/dtourolle/KPN" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
dtourolle/KPN
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/." class="md-nav__link">
<span class="md-ellipsis">
Home
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/getting-started/" class="md-nav__link">
<span class="md-ellipsis">
Getting Started
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3" id="__nav_3_label" tabindex="0">
<span class="md-ellipsis">
Concepts
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
Concepts
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/nodes/" class="md-nav__link">
<span class="md-ellipsis">
Nodes
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/network/" class="md-nav__link">
<span class="md-ellipsis">
Networks
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/channels/" class="md-nav__link">
<span class="md-ellipsis">
Channels
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="/error-handling/" class="md-nav__link">
<span class="md-ellipsis">
Error Handling & Events
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_5" >
<label class="md-nav__link" for="__nav_5" id="__nav_5_label" tabindex="0">
<span class="md-ellipsis">
Advanced
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_5_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_5">
<span class="md-nav__icon md-icon"></span>
Advanced
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/static-network/" class="md-nav__link">
<span class="md-ellipsis">
Static Networks
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/shared-resource/" class="md-nav__link">
<span class="md-ellipsis">
Shared Resources
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/fanout/" class="md-nav__link">
<span class="md-ellipsis">
Fan-out & Routing
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="/examples/" class="md-nav__link">
<span class="md-ellipsis">
Examples
</span>
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1>404 - Not found</h1>
</article>
</div>
<script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script>
</div>
<button type="button" class="md-top md-icon" data-md-component="top" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg>
Back to top
</button>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"annotate": null, "base": "/", "features": ["navigation.tabs", "navigation.sections", "navigation.top", "content.code.copy", "content.code.annotate"], "search": "/assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
<script src="/assets/javascripts/bundle.79ae519e.min.js"></script>
</body>
</html>
+112
View File
@@ -0,0 +1,112 @@
cmake_minimum_required(VERSION 3.21)
project(kpnpp VERSION 0.1.0 LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
option(KPN_BUILD_TESTS "Build tests" ON)
option(KPN_BUILD_PYTHON "Build Python bindings (requires nanobind)" ON)
option(KPN_BUILD_EXAMPLES "Build examples" ON)
option(KPN_WEB_DEBUG "Enable web debug UI (cpp-httplib)" OFF)
# Sanitizer build. Empty = off. Accepts "thread", "address", "undefined",
# or a combination like "address,undefined". Applied to all kpn targets via
# the kpn_sanitizer_flags() helper below.
#
# The lock-free SPSC Channel<T> (include/kpn/channel.hpp) has hand-reasoned
# acquire/release ordering; -DKPN_SANITIZER=thread + the channel stress test
# (tests/test_channel_stress.cpp) is the dynamic half of verifying it. The
# static half is the CDSChecker model-check harness (see verify/).
set(KPN_SANITIZER "" CACHE STRING
"Build with sanitizer: thread | address | undefined | <combo> (empty = off)")
# Translate KPN_SANITIZER into compile/link flags. No-op when empty.
function(kpn_sanitizer_flags out_var)
if(KPN_SANITIZER)
set(${out_var}
-fsanitize=${KPN_SANITIZER}
-fno-omit-frame-pointer
-g
PARENT_SCOPE)
else()
set(${out_var} "" PARENT_SCOPE)
endif()
endfunction()
# ── Core library (header-only) ────────────────────────────────────────────────
add_library(kpn INTERFACE)
target_include_directories(kpn INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>
)
target_compile_features(kpn INTERFACE cxx_std_20)
# Threads required by node/channel implementation
find_package(Threads REQUIRED)
target_link_libraries(kpn INTERFACE Threads::Threads)
# ── Web debug UI (optional) ───────────────────────────────────────────────────
if(KPN_WEB_DEBUG)
include(FetchContent)
FetchContent_Declare(
cpp-httplib
GIT_REPOSITORY https://github.com/yhirose/cpp-httplib.git
GIT_TAG v0.18.0
)
FetchContent_MakeAvailable(cpp-httplib)
# httplib made available but NOT forced onto kpn interface — targets opt in
# by defining KPN_WEB_DEBUG=1 and linking httplib::httplib themselves.
# This prevents tests and other examples from pulling in the HTTP server.
endif()
# Convenience function for targets that want web debug
function(kpn_target_enable_web_debug target)
target_compile_definitions(${target} PRIVATE KPN_WEB_DEBUG=1)
target_link_libraries(${target} PRIVATE httplib::httplib)
endfunction()
# ── Tests ─────────────────────────────────────────────────────────────────────
if(KPN_BUILD_TESTS)
enable_testing()
add_subdirectory(tests)
endif()
# ── Benchmarks ────────────────────────────────────────────────────────────────
option(KPN_BUILD_BENCHMARKS "Build benchmarks" OFF)
if(KPN_BUILD_BENCHMARKS)
add_subdirectory(benchmarks)
endif()
# ── Python bindings ───────────────────────────────────────────────────────────
if(KPN_BUILD_PYTHON)
find_package(Python 3.8 COMPONENTS Interpreter Development.Module REQUIRED)
find_package(nanobind CONFIG QUIET)
if(NOT nanobind_FOUND)
# Fall back to FetchContent if nanobind not installed system-wide
include(FetchContent)
FetchContent_Declare(
nanobind
GIT_REPOSITORY https://github.com/wjakob/nanobind.git
GIT_TAG v2.12.0
)
FetchContent_MakeAvailable(nanobind)
endif()
add_subdirectory(python)
endif()
# ── Examples ──────────────────────────────────────────────────────────────────
if(KPN_BUILD_EXAMPLES)
add_subdirectory(examples)
endif()
# ── Docs (README generation) ──────────────────────────────────────────────────
find_package(Python3 QUIET COMPONENTS Interpreter)
if(Python3_FOUND)
add_custom_target(docs
COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/scripts/render_readme.py
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMENT "Rendering README.md from README.md.in"
VERBATIM
)
endif()
+27
View File
@@ -0,0 +1,27 @@
# KPN++ Builder Image (CI: pipeline trigger v2)
# Pre-built image with GCC, CMake, Ninja, and Python dev headers for building and testing KPN++
# Build: docker build -f Dockerfile.builder -t gitea.tourolle.paris/dtourolle/kpnpp-builder:latest .
# Push: docker push gitea.tourolle.paris/dtourolle/kpnpp-builder:latest
FROM gcc:14
RUN apt-get update && apt-get install -y --no-install-recommends \
cmake \
ninja-build \
python3 \
python3-dev \
python3-pip \
git \
ca-certificates \
nodejs \
&& rm -rf /var/lib/apt/lists/*
# Pre-install MkDocs dependencies so the docs workflow does not need to pip
# install at runtime. --break-system-packages is required because the Debian
# base marks the environment as externally managed (PEP 668); this is safe in
# a dedicated container image.
COPY docs/requirements.txt /tmp/docs-requirements.txt
RUN pip install --no-cache-dir --break-system-packages -r /tmp/docs-requirements.txt \
&& rm /tmp/docs-requirements.txt
WORKDIR /src
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Duncan Tourolle
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+612
View File
@@ -0,0 +1,612 @@
# KPN++
A C++20 Kahn Process Network (KPN) library. Each node wraps a function and runs in its own thread, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind.
📖 **[Documentation](https://pages.tourolle.paris/dtourolle/kpn/)**
---
## Requirements
| Dependency | Version | Notes |
|---|---|---|
| CMake | ≥ 3.21 | |
| C++ compiler | GCC ≥ 11, Clang ≥ 13, MSVC 19.29 | C++20 required |
| Threads | system | `find_package(Threads)` |
| nanobind | ≥ 2.1 | auto-fetched if not installed; Python ≥ 3.8 |
| Catch2 | v3 | auto-fetched for tests |
| Google Test | v1.14 | auto-fetched for tests |
| OpenCV | ≥ 4 | optional; only for example 09 |
---
## Build
```bash
cmake -B build -DKPN_BUILD_PYTHON=OFF # core + tests + C++ examples
cmake --build build --parallel
ctest --test-dir build
```
Enable Python bindings (requires nanobind and Python dev headers):
```bash
cmake -B build -DKPN_BUILD_PYTHON=ON
cmake --build build --parallel
```
Disable examples:
```bash
cmake -B build -DKPN_BUILD_EXAMPLES=OFF
```
---
## Core Concepts
### Nodes
A node wraps any callable. Its input types are taken from the function's parameter list; its output types from the return type. Multi-output nodes return `std::tuple<...>`.
```cpp
#include <kpn/kpn.hpp>
using namespace kpn;
```
Source, transform, and sink — from [`examples/01_hello_pipeline/main.cpp`](examples/01_hello_pipeline/main.cpp):
```cpp
static int produce() { return 42; }
static int double_it(int x) { return x * 2; }
static void print_it(int x) { std::cout << "result: " << x << '\n'; }
```
Multi-output node returning a tuple — from [`examples/03_multi_output/main.cpp`](examples/03_multi_output/main.cpp):
```cpp
// Multi-output: returns (key, value) as a tuple — KPN++ routes each element
// to its own output port automatically.
static std::tuple<std::string, std::string> parse(std::string kv) {
auto sep = kv.find('=');
if (sep == std::string::npos) return {kv, ""};
return {kv.substr(0, sep), kv.substr(sep + 1)};
}
```
### Creating Nodes
**Index-only ports** (from [`examples/01_hello_pipeline/main.cpp`](examples/01_hello_pipeline/main.cpp)):
```cpp
auto src = make_node<produce>(5);
auto dbl = make_node<double_it>(5);
auto sink = make_node<print_it>(5);
```
**Named ports** (from [`examples/02_named_ports/main.cpp`](examples/02_named_ports/main.cpp)):
```cpp
// tokenise: no inputs, one named output "words"
auto tok = make_node<tokenise>(out<"words">{}, 4);
// count_words: named input "words", named outputs "count" and "words"
auto cnt = make_node<count_words>(in<"words">{}, out<"count", "words">{}, 4);
// report: two named inputs
auto snk = make_node<report>(in<"count", "words">{}, 4);
```
**Multi-named output source** (from [`examples/09_opencv_cellshade/main.cpp`](examples/09_opencv_cellshade/main.cpp)):
```cpp
auto src = make_node<capture>(out<"colour","grey">{}, 8);
```
Port names are NTTP `fixed_string` values — resolved entirely at compile time, zero runtime cost.
### Building a Network
`Network` is **non-owning** — declare nodes first, then register them. Nodes must outlive the network.
From [`examples/02_named_ports/main.cpp`](examples/02_named_ports/main.cpp):
```cpp
Network net;
net.add("tok", tok)
.add("cnt", cnt)
.add("snk", snk)
.connect("tok", tok.template output<"words">(), "cnt", cnt.template input<"words">())
.connect("cnt", cnt.template output<"count">(), "snk", snk.template input<"count">())
.connect("cnt", cnt.template output<"words">(), "snk", snk.template input<"words">())
.build();
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(500));
net.stop();
```
`.build()` runs cycle detection — throws `NetworkCycleError` on cycles.
> **Named port syntax in template context:** when the node variable is `auto`-deduced, use `.template output<"name">()` and `.template input<"name">()` to help the parser.
### Channel Semantics
- **Bounded FIFO**: default capacity 5, configurable per-node at construction.
- **Blocking `pop()`**: consumer blocks until data is available (KPN semantics).
- **Throwing `push()`**: throws `ChannelOverflowError` if the channel is full and accepting.
- **Silent drop on disabled channel**: after `node.stop()`, its input channels are disabled — producers that push into them have the value silently dropped. No exception, no blocking.
- **Source throttling**: source nodes (no inputs) must sleep or yield to avoid overflowing downstream FIFOs. See example 09.
### Storage Policy
Large types (`sizeof > 8` or non-trivially-copyable) are stored as `std::shared_ptr<const T>` inside the channel — no copies, shared immutable ownership. Small trivially-copyable types are stored by value.
Override the policy for a specific type (from [`examples/04_storage_policy/main.cpp`](examples/04_storage_policy/main.cpp)):
```cpp
// Override: store Tag by value despite being a struct
// (it's trivially copyable and small — this just makes the policy explicit)
template<>
struct kpn::channel_storage_policy<Tag> {
static constexpr bool by_value = true;
};
```
### Diagnostics & Error Handling
Custom diagnostics handler — fires on the watchdog interval (from [`examples/05_error_handling/main.cpp`](examples/05_error_handling/main.cpp)):
```cpp
// Custom diagnostics handler — fires on the watchdog interval.
// Print a concise one-liner rather than the full table.
net.set_diagnostics_handler([](const std::vector<NodeSnapshot>& nodes,
const std::vector<ChannelSnapshot>& channels) {
std::cout << "[diag] ";
for (auto& n : nodes)
std::cout << n.name << "=" << n.throughput_fps << "fps ";
for (auto& c : channels)
std::cout << "channel fill=" << static_cast<int>(c.fill_pct()) << "% "
<< "overflows=" << c.overflows;
std::cout << '\n';
});
```
### Shutdown
`node.stop()` / `net.stop()`:
1. Sets `accepting_ = false` on all input channels (drops in-flight pushes silently).
2. Clears any queued items from those channels.
3. Unblocks any thread blocked on `pop()` (throws `ChannelClosedError` inside `run_loop`, which exits cleanly).
4. Joins the node thread.
---
## Named Ports — Design Notes
Port names use C++20 NTTP `fixed_string`. The deduction guide is required:
```cpp
template<std::size_t N>
fixed_string(const char (&)[N]) -> fixed_string<N>;
```
`fixed_string<4>` and `fixed_string<7>` are distinct types — `input<"img">()` and `input<"sigma">()` resolve to different template instantiations at compile time. Wrong names produce a `static_assert` at the call site with a readable message.
---
## Sub-Networks
`Network` implements `INode`, so it can be nested inside a larger `Network`:
```cpp
// Inner sub-network
Network pipe;
pipe.add("pre", pre_node)
.add("enh", enh_node)
.connect("pre", pre_node.output<0>(), "enh", enh_node.input<0>())
.expose_input("img", pre_node.input<0>())
.expose_output("result", enh_node.output<0>())
.build();
// Outer network
Network top;
top.add("pipe", pipe)
.add("sink", sink_node)
.connect("pipe", pipe.output<"result">(), "sink", sink_node.input<0>())
.build();
top.start();
```
---
## Display / GUI Nodes
**Do not wrap `imshow`/`waitKey` as a KPN node.** Qt and Wayland require these to run on the main thread (the thread that owns the event loop). Instead, derive from `MainThreadNode<>` — it owns the input channels, implements `INode`, and exposes a `step()` method to call on the main thread.
`DisplayNode` from [`examples/09_opencv_cellshade/main.cpp`](examples/09_opencv_cellshade/main.cpp):
```cpp
class DisplayNode : public kpn::MainThreadNode<DisplayNode,
kpn::in<"composite", "edges">,
cv::Mat, cv::Mat> {
public:
DisplayNode() : MainThreadNode(8) {
cv::namedWindow("Cell Shade", cv::WINDOW_NORMAL);
cv::namedWindow("Edge Mask", cv::WINDOW_NORMAL);
cv::resizeWindow("Cell Shade", 1280, 720);
cv::resizeWindow("Edge Mask", 640, 360);
}
~DisplayNode() { cv::destroyAllWindows(); }
bool operator()(cv::Mat composite, cv::Mat edges) {
cv::imshow("Cell Shade", composite);
cv::Mat edges_bgr;
cv::cvtColor(edges, edges_bgr, cv::COLOR_GRAY2BGR);
cv::imshow("Edge Mask", edges_bgr);
int key = cv::waitKey(1);
if (key == 'q' || key == 27) return false;
return window_open("Cell Shade") && window_open("Edge Mask");
}
private:
static bool window_open(const char* name) {
try { return cv::getWindowProperty(name, cv::WND_PROP_VISIBLE) >= 1; }
catch (const cv::Exception&) { return false; }
}
};
```
Wire it into the network and drive it from the main thread:
```cpp
net.start();
// Main thread drives display — imshow/waitKey stay on the GUI thread.
// step() returns false when operator() returns false (q pressed / window closed).
while (disp.step())
cv::waitKey(8); // yield event loop when no frame ready
net.stop();
```
---
## OpenCV Cell-Shading Example
Real-time cell-shading pipeline from [`examples/09_opencv_cellshade/main.cpp`](examples/09_opencv_cellshade/main.cpp).
**Source node** — returns two frames (colour + grey) as a tuple, routing them to separate downstream branches:
```cpp
static std::tuple<cv::Mat, cv::Mat> capture() {
constexpr int W = 640, H = 480;
static cv::VideoCapture cap;
static bool opened = false;
if (!opened) {
opened = true;
cap.open(0, cv::CAP_V4L2);
if (cap.isOpened()) {
cap.set(cv::CAP_PROP_FRAME_WIDTH, W);
cap.set(cv::CAP_PROP_FRAME_HEIGHT, H);
} else {
std::cerr << "[capture] no webcam — using synthetic animated pattern\n";
}
}
cv::Mat frame;
if (cap.isOpened()) {
auto t0 = std::chrono::steady_clock::now();
cap >> frame;
auto elapsed = std::chrono::steady_clock::now() - t0;
if (elapsed < std::chrono::milliseconds(20))
std::this_thread::sleep_for(std::chrono::milliseconds(33) - elapsed);
if (frame.empty()) frame = cv::Mat::zeros(H, W, CV_8UC3);
} else {
static int tick = 0;
static cv::Mat grad = make_gradient(W, H);
++tick;
frame = grad.clone();
int r = 150 + (tick % 80) * 4;
cv::circle(frame, {W/2, H/2}, r, {255, 200, 0}, -1);
cv::circle(frame, {W/2, H/2}, r / 2, { 0, 128, 255}, -1);
cv::circle(frame, {W*2/5, H*2/5}, r / 3, {200, 0, 200}, -1);
std::this_thread::sleep_for(std::chrono::milliseconds(33));
}
return {frame.clone(), frame.clone()};
}
```
**Full network wiring:**
```cpp
auto src = make_node<capture> (out<"colour","grey">{}, 8);
auto gray_node = make_node<to_gray> (in<"bgr">{}, out<"gray">{}, 8);
auto edge_node = make_node<edges_fn> (in<"gray">{}, out<"edges">{}, 8);
auto quant = make_node<quantise> (in<"bgr">{}, out<"quantised">{}, 8);
auto comp = make_node<composite>(in<"edges","colour">{}, out<"result","edges">{}, 8);
// DisplayNode: two windows opened in constructor, step() drives main thread.
DisplayNode disp;
Network net;
net.add("src", src)
.add("gray", gray_node)
.add("edges", edge_node)
.add("quant", quant)
.add("comp", comp)
.add("display", disp)
.connect("src", src.template output<"colour">(), "quant", quant.template input<"bgr">())
.connect("quant", quant.template output<"quantised">(), "comp", comp.template input<"colour">())
.connect("src", src.template output<"grey">(), "gray", gray_node.template input<"bgr">())
.connect("gray", gray_node.template output<"gray">(), "edges", edge_node.template input<"gray">())
.connect("edges", edge_node.template output<"edges">(), "comp", comp.template input<"edges">())
.connect("comp", comp.template output<"result">(), "display", disp.template input<"composite">())
.connect("comp", comp.template output<"edges">(), "display", disp.template input<"edges">())
.build();
```
---
## Fan-Out (Multi-Output)
From [`examples/03_multi_output/main.cpp`](examples/03_multi_output/main.cpp) — one node fans out to two independent sinks via a tuple return:
```cpp
auto gen = make_node<generate>(out<"kv">{}, 4);
auto par = make_node<parse> (in<"kv">{}, out<"key", "value">{}, 4);
auto keys = make_node<print_key> (in<"key">{}, 4);
auto vals = make_node<print_value>(in<"value">{}, 4);
Network net;
net.add("gen", gen)
.add("par", par)
.add("keys", keys)
.add("vals", vals)
.connect("gen", gen.template output<"kv">(), "par", par.template input<"kv">())
.connect("par", par.template output<"key">(), "keys", keys.template input<"key">())
.connect("par", par.template output<"value">(), "vals", vals.template input<"value">())
.build();
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(600));
net.stop();
```
---
## Python Bindings
> Python bindings are scaffolded but not yet fully implemented. See `python/kpn_python.cpp` and `include/kpn/python/bindings.hpp`.
A `PyNetwork` is constructed from a closed list of C++ node types. The variant of all port types is derived at compile time — no runtime type registration needed.
**GIL rules (non-negotiable):**
- Acquire the GIL only for the duration of a Python callable invocation.
- Release the GIL before any blocking channel operation (`pop()`, `push()`, `net.read()`, `net.write()`).
Violating the second rule deadlocks.
---
## Examples
| Example | What it shows |
|---|---|
| `01_hello_pipeline` | Linear pipeline, index-based port wiring |
| `02_named_ports` | `in<>`/`out<>` name tags, named port access |
| `03_multi_output` | Tuple-returning node, per-element sub-port routing |
| `04_storage_policy` | `channel_storage_policy` default and specialisation |
| `05_error_handling` | `ChannelOverflowError`, `ErrorHandler` |
| `06_watchdog` | Watchdog interval, stall detection |
| `07_python_network` | PyNetwork with a pure-Python node between a C++ source and sink |
| `08_python_subport` | Drive a Python node from Python via `net.write`/`net.read` sub-port taps |
| `09_opencv_cellshade` | Real-time cell-shading on webcam/pattern; requires OpenCV ≥ 4 |
Run the cell-shading example:
```bash
./build/examples/09_opencv_cellshade
# Press 'q' or close the window to stop.
# Falls back to an animated synthetic pattern if no webcam is found.
```
---
## Performance
Measured on Linux (x86-64, `-O3 -march=native`) with `benchmarks/bench_pipeline`.
Each topology pushes N items through the graph; `overhead_us/item` strips out the
per-node compute time to isolate framework cost.
Overhead formula: `(elapsed (N + depth 1) × work_us) / N` removes the expected
pipeline-fill cost so the number reflects pure framework latency.
### Baseline overhead (private pools, 100 µs/node)
| Topology | items/sec | overhead µs/item |
|---|---|---|
| chain depth-1 | 9 797 | ~2 |
| chain depth-4 | 9 448 | ~4 |
| chain depth-8 | 9 078 | ~7 |
| chain depth-16 | 7 004 | ~13 ← oversubscription |
| chain depth-32 | 4 179 | ~77 ← oversubscription |
| wide fanout-1 | 9 751 | ~3 |
| wide fanout-4 | 9 668 | ~3 |
| diamond (2×2) | 9 607 | ~4 |
Chain overhead is flat at **~27 µs/hop** for depths within the machine's core count,
then rises once threads compete for CPU. Wide and diamond topologies add no measurable
overhead as fanout increases — all branches run in parallel.
### Scheduling modes
`Node<>` gives each node a private `ThreadPool(1)`. `PoolNode<>` lets multiple
nodes share one pool. The right choice depends on the graph shape:
| Scenario | Recommended |
|---|---|
| Work per node < 100 µs, deep chain | Private pools — lower per-hop latency |
| Work per node ≥ 100 µs, wide/diamond | Shared pool, `threads = hardware_concurrency` |
| Any graph, bounded thread count required | Shared pool, `threads ≥ max parallel nodes` |
A shared single-thread pool (`threads=1`) fully serialises the graph — throughput
divides by depth for chains and by width for fanout topologies. A shared pool with
`threads ≥ max_concurrent_nodes` matches private-pool throughput while keeping the
OS thread count bounded.
### vs. TBB flow graph
Benchmarked against `tbb::flow::function_node<int,int>` (serial concurrency) with
`tbb::flow::broadcast_node<int>` for fanout. Run with `cmake -DKPN_BUILD_BENCHMARKS=ON`
— TBB benchmarks are included automatically when `find_package(TBB)` succeeds.
**Channel implementation:** lock-free SPSC ring buffer with `std::atomic::wait/notify_one`
(C++20 portable futex) plus a configurable spin-before-sleep window (default ~4 µs).
Large types are stored as `shared_ptr<const T>` — fanout copies reference counts,
not data.
Overhead µs/item at **work_us = 10** (framework overhead dominates):
| Topology | KPN++ | TBB |
|---|---|---|
| chain depth-1 | 1.7 | **1.4** |
| chain depth-4 | 2.5 | **2.2** |
| chain depth-8 | **3.0** | 3.6 |
| chain depth-16 | **9.3** | 13.0 |
| chain depth-32 | 23.2 | **14.2** |
| wide fanout-4 | 2.5 | **1.4** |
| diamond (2×2) | 3.4 | **1.9** |
Overhead µs/item at **work_us = 100** (moderate compute, KPN wins):
| Topology | KPN++ | TBB |
|---|---|---|
| chain depth-1 | **2.1** | 3.5 |
| chain depth-4 | **4.3** | 5.2 |
| chain depth-8 | **6.7** | 8.5 |
| chain depth-16 | **12.8** | 17.4 |
| chain depth-32 | **77** | 81 |
| wide fanout-4 | 3.4 | **1.9** |
| diamond (2×2) | **4.1** | 6.1 |
KPN++ pools beat TBB for every chain and diamond topology at 100 µs/node, and
match TBB within ~20% at 10 µs/node for shallow chains. TBB retains an edge on wide
fanout (serial dispatch loop vs. work-stealing pool) and at extreme oversubscription
depths (chain-32 at 10 µs). The remaining gap at light work is the cost of
`atomic::wait` vs. TBB's continuously-spinning worker threads.
### vs. TBB — API
The function signature is the node. KPN infers input and output types automatically;
there is no graph object to manage.
**Single-output node:**
```cpp
// KPN — 1 line
int scale(int x) { return x * 2; }
// TBB — must state types, concurrency policy, and carry a graph reference
tbb::flow::function_node<int,int> n(g, tbb::flow::serial, [](int x){ return x*2; });
```
**Multi-output node:**
```cpp
// KPN — return a tuple
std::tuple<cv::Mat,cv::Mat> split(cv::Mat f) { return {f, f}; }
// TBB — multifunction_node + explicit try_put per port
tbb::flow::multifunction_node<cv::Mat, std::tuple<cv::Mat,cv::Mat>> n(
g, tbb::flow::serial,
[](cv::Mat f, auto& ports) {
std::get<0>(ports).try_put(f);
std::get<1>(ports).try_put(f);
});
```
**Named ports** — compile-time checked, zero runtime cost, not available in TBB:
```cpp
auto node = make_node<split>(in<"frame">{}, out<"colour","grey">{}, 5);
net.connect("cam", cam.output<"frame">(), "split", node.input<"frame">());
// ^^^^^^^ typo → compile error
```
| | KPN | TBB |
|---|---|---|
| Node definition | plain function | `function_node<In,Out>` + explicit types |
| Multi-output | `return std::tuple<A,B>` | `multifunction_node` + `try_put` × N |
| Named ports | `in<"name">` / `out<"name">` compile-time | none |
| Graph lifetime | none | `graph g` must outlive all nodes |
| Shutdown | `net.stop()` | `g.wait_for_all()` + manual |
| Python bindings | designed-in | none |
Build the benchmarks with:
```bash
cmake -B build -DKPN_BUILD_BENCHMARKS=ON
cmake --build build --target bench_pipeline
./build/benchmarks/bench_pipeline | tee results.csv
```
---
## Project Structure
```
include/kpn/
fixed_string.hpp — NTTP string, in<>/out<> tags, index_of
traits.hpp — function_traits, normalised_return_t, output_count_v
channel.hpp — Channel<T>, channel_storage_policy, exceptions
port.hpp — InputPort<N,I>, OutputPort<N,I>
node.hpp — Node<Func,in<...>,out<...>>, make_node, INode
network.hpp — Network (builder, cycle detection, watchdog)
variant_node.hpp — VariantNode, PythonConverter<T>, unique_types (Python layer)
python/
bindings.hpp — nanobind helpers, GIL rule documentation
kpn.hpp — umbrella header
src/
network.cpp — non-template Network implementation
tests/
test_fixed_string.cpp
test_traits.cpp
test_channel.cpp
test_node.cpp
test_network.cpp
python/
kpn_python.cpp — nanobind module entry point
examples/
01_hello_pipeline/ … 09_opencv_cellshade/
scripts/
render_readme.py — regenerates README.md from README.md.in
```
---
## Contributing
Contributions are welcome. This project is hosted on a self-hosted Gitea
instance that accepts sign-in and registration with a GitHub account, so you
can log in with your existing GitHub identity to open issues and pull requests.
If you change any code that appears in a README snippet, edit `README.md.in`
(the template) rather than `README.md` directly, then regenerate:
```bash
cmake --build build --target readme # or: python scripts/render_readme.py
```
---
## Acknowledgments
AI tooling was used heavily throughout the development of this project,
including the design, implementation, tests, and documentation. All output
has been reviewed, but please keep this in mind when reading or building on the
code.
---
## License
Released under the [MIT License](LICENSE). Copyright (c) 2026 Duncan Tourolle.
+434
View File
@@ -0,0 +1,434 @@
# KPN++
A C++20 Kahn Process Network (KPN) library. Each node wraps a function and runs in its own thread, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind.
📖 **[Documentation](https://pages.tourolle.paris/dtourolle/kpn/)**
---
## Requirements
| Dependency | Version | Notes |
|---|---|---|
| CMake | ≥ 3.21 | |
| C++ compiler | GCC ≥ 11, Clang ≥ 13, MSVC 19.29 | C++20 required |
| Threads | system | `find_package(Threads)` |
| nanobind | ≥ 2.1 | auto-fetched if not installed; Python ≥ 3.8 |
| Catch2 | v3 | auto-fetched for tests |
| Google Test | v1.14 | auto-fetched for tests |
| OpenCV | ≥ 4 | optional; only for example 09 |
---
## Build
```bash
cmake -B build -DKPN_BUILD_PYTHON=OFF # core + tests + C++ examples
cmake --build build --parallel
ctest --test-dir build
```
Enable Python bindings (requires nanobind and Python dev headers):
```bash
cmake -B build -DKPN_BUILD_PYTHON=ON
cmake --build build --parallel
```
Disable examples:
```bash
cmake -B build -DKPN_BUILD_EXAMPLES=OFF
```
---
## Core Concepts
### Nodes
A node wraps any callable. Its input types are taken from the function's parameter list; its output types from the return type. Multi-output nodes return `std::tuple<...>`.
```cpp
#include <kpn/kpn.hpp>
using namespace kpn;
```
Source, transform, and sink — from [`examples/01_hello_pipeline/main.cpp`](examples/01_hello_pipeline/main.cpp):
<!-- @snippet examples/01_hello_pipeline/main.cpp basic_node_fns -->
Multi-output node returning a tuple — from [`examples/03_multi_output/main.cpp`](examples/03_multi_output/main.cpp):
<!-- @snippet examples/03_multi_output/main.cpp multi_output_fn -->
### Creating Nodes
**Index-only ports** (from [`examples/01_hello_pipeline/main.cpp`](examples/01_hello_pipeline/main.cpp)):
<!-- @snippet examples/01_hello_pipeline/main.cpp index_only_nodes -->
**Named ports** (from [`examples/02_named_ports/main.cpp`](examples/02_named_ports/main.cpp)):
<!-- @snippet examples/02_named_ports/main.cpp named_port_creation -->
**Multi-named output source** (from [`examples/09_opencv_cellshade/main.cpp`](examples/09_opencv_cellshade/main.cpp)):
```cpp
auto src = make_node<capture>(out<"colour","grey">{}, 8);
```
Port names are NTTP `fixed_string` values — resolved entirely at compile time, zero runtime cost.
### Building a Network
`Network` is **non-owning** — declare nodes first, then register them. Nodes must outlive the network.
From [`examples/02_named_ports/main.cpp`](examples/02_named_ports/main.cpp):
<!-- @snippet examples/02_named_ports/main.cpp named_port_network -->
`.build()` runs cycle detection — throws `NetworkCycleError` on cycles.
> **Named port syntax in template context:** when the node variable is `auto`-deduced, use `.template output<"name">()` and `.template input<"name">()` to help the parser.
### Channel Semantics
- **Bounded FIFO**: default capacity 5, configurable per-node at construction.
- **Blocking `pop()`**: consumer blocks until data is available (KPN semantics).
- **Throwing `push()`**: throws `ChannelOverflowError` if the channel is full and accepting.
- **Silent drop on disabled channel**: after `node.stop()`, its input channels are disabled — producers that push into them have the value silently dropped. No exception, no blocking.
- **Source throttling**: source nodes (no inputs) must sleep or yield to avoid overflowing downstream FIFOs. See example 09.
### Storage Policy
Large types (`sizeof > 8` or non-trivially-copyable) are stored as `std::shared_ptr<const T>` inside the channel — no copies, shared immutable ownership. Small trivially-copyable types are stored by value.
Override the policy for a specific type (from [`examples/04_storage_policy/main.cpp`](examples/04_storage_policy/main.cpp)):
<!-- @snippet examples/04_storage_policy/main.cpp storage_policy_spec -->
### Diagnostics & Error Handling
Custom diagnostics handler — fires on the watchdog interval (from [`examples/05_error_handling/main.cpp`](examples/05_error_handling/main.cpp)):
<!-- @snippet examples/05_error_handling/main.cpp diagnostics_handler -->
### Shutdown
`node.stop()` / `net.stop()`:
1. Sets `accepting_ = false` on all input channels (drops in-flight pushes silently).
2. Clears any queued items from those channels.
3. Unblocks any thread blocked on `pop()` (throws `ChannelClosedError` inside `run_loop`, which exits cleanly).
4. Joins the node thread.
---
## Named Ports — Design Notes
Port names use C++20 NTTP `fixed_string`. The deduction guide is required:
```cpp
template<std::size_t N>
fixed_string(const char (&)[N]) -> fixed_string<N>;
```
`fixed_string<4>` and `fixed_string<7>` are distinct types — `input<"img">()` and `input<"sigma">()` resolve to different template instantiations at compile time. Wrong names produce a `static_assert` at the call site with a readable message.
---
## Sub-Networks
`Network` implements `INode`, so it can be nested inside a larger `Network`:
```cpp
// Inner sub-network
Network pipe;
pipe.add("pre", pre_node)
.add("enh", enh_node)
.connect("pre", pre_node.output<0>(), "enh", enh_node.input<0>())
.expose_input("img", pre_node.input<0>())
.expose_output("result", enh_node.output<0>())
.build();
// Outer network
Network top;
top.add("pipe", pipe)
.add("sink", sink_node)
.connect("pipe", pipe.output<"result">(), "sink", sink_node.input<0>())
.build();
top.start();
```
---
## Display / GUI Nodes
**Do not wrap `imshow`/`waitKey` as a KPN node.** Qt and Wayland require these to run on the main thread (the thread that owns the event loop). Instead, derive from `MainThreadNode<>` — it owns the input channels, implements `INode`, and exposes a `step()` method to call on the main thread.
`DisplayNode` from [`examples/09_opencv_cellshade/main.cpp`](examples/09_opencv_cellshade/main.cpp):
<!-- @snippet examples/09_opencv_cellshade/main.cpp display_node -->
Wire it into the network and drive it from the main thread:
<!-- @snippet examples/09_opencv_cellshade/main.cpp main_thread_step -->
---
## OpenCV Cell-Shading Example
Real-time cell-shading pipeline from [`examples/09_opencv_cellshade/main.cpp`](examples/09_opencv_cellshade/main.cpp).
**Source node** — returns two frames (colour + grey) as a tuple, routing them to separate downstream branches:
<!-- @snippet examples/09_opencv_cellshade/main.cpp capture_fn -->
**Full network wiring:**
<!-- @snippet examples/09_opencv_cellshade/main.cpp opencv_network -->
---
## Fan-Out (Multi-Output)
From [`examples/03_multi_output/main.cpp`](examples/03_multi_output/main.cpp) — one node fans out to two independent sinks via a tuple return:
<!-- @snippet examples/03_multi_output/main.cpp fanout_network -->
---
## Python Bindings
> Python bindings are scaffolded but not yet fully implemented. See `python/kpn_python.cpp` and `include/kpn/python/bindings.hpp`.
A `PyNetwork` is constructed from a closed list of C++ node types. The variant of all port types is derived at compile time — no runtime type registration needed.
**GIL rules (non-negotiable):**
- Acquire the GIL only for the duration of a Python callable invocation.
- Release the GIL before any blocking channel operation (`pop()`, `push()`, `net.read()`, `net.write()`).
Violating the second rule deadlocks.
---
## Examples
| Example | What it shows |
|---|---|
| `01_hello_pipeline` | Linear pipeline, index-based port wiring |
| `02_named_ports` | `in<>`/`out<>` name tags, named port access |
| `03_multi_output` | Tuple-returning node, per-element sub-port routing |
| `04_storage_policy` | `channel_storage_policy` default and specialisation |
| `05_error_handling` | `ChannelOverflowError`, `ErrorHandler` |
| `06_watchdog` | Watchdog interval, stall detection |
| `07_python_network` | PyNetwork with a pure-Python node between a C++ source and sink |
| `08_python_subport` | Drive a Python node from Python via `net.write`/`net.read` sub-port taps |
| `09_opencv_cellshade` | Real-time cell-shading on webcam/pattern; requires OpenCV ≥ 4 |
Run the cell-shading example:
```bash
./build/examples/09_opencv_cellshade
# Press 'q' or close the window to stop.
# Falls back to an animated synthetic pattern if no webcam is found.
```
---
## Performance
Measured on Linux (x86-64, `-O3 -march=native`) with `benchmarks/bench_pipeline`.
Each topology pushes N items through the graph; `overhead_us/item` strips out the
per-node compute time to isolate framework cost.
Overhead formula: `(elapsed (N + depth 1) × work_us) / N` removes the expected
pipeline-fill cost so the number reflects pure framework latency.
### Baseline overhead (private pools, 100 µs/node)
| Topology | items/sec | overhead µs/item |
|---|---|---|
| chain depth-1 | 9 797 | ~2 |
| chain depth-4 | 9 448 | ~4 |
| chain depth-8 | 9 078 | ~7 |
| chain depth-16 | 7 004 | ~13 ← oversubscription |
| chain depth-32 | 4 179 | ~77 ← oversubscription |
| wide fanout-1 | 9 751 | ~3 |
| wide fanout-4 | 9 668 | ~3 |
| diamond (2×2) | 9 607 | ~4 |
Chain overhead is flat at **~27 µs/hop** for depths within the machine's core count,
then rises once threads compete for CPU. Wide and diamond topologies add no measurable
overhead as fanout increases — all branches run in parallel.
### Scheduling modes
`Node<>` gives each node a private `ThreadPool(1)`. `PoolNode<>` lets multiple
nodes share one pool. The right choice depends on the graph shape:
| Scenario | Recommended |
|---|---|
| Work per node < 100 µs, deep chain | Private pools — lower per-hop latency |
| Work per node ≥ 100 µs, wide/diamond | Shared pool, `threads = hardware_concurrency` |
| Any graph, bounded thread count required | Shared pool, `threads ≥ max parallel nodes` |
A shared single-thread pool (`threads=1`) fully serialises the graph — throughput
divides by depth for chains and by width for fanout topologies. A shared pool with
`threads ≥ max_concurrent_nodes` matches private-pool throughput while keeping the
OS thread count bounded.
### vs. TBB flow graph
Benchmarked against `tbb::flow::function_node<int,int>` (serial concurrency) with
`tbb::flow::broadcast_node<int>` for fanout. Run with `cmake -DKPN_BUILD_BENCHMARKS=ON`
— TBB benchmarks are included automatically when `find_package(TBB)` succeeds.
**Channel implementation:** lock-free SPSC ring buffer with `std::atomic::wait/notify_one`
(C++20 portable futex) plus a configurable spin-before-sleep window (default ~4 µs).
Large types are stored as `shared_ptr<const T>` — fanout copies reference counts,
not data.
Overhead µs/item at **work_us = 10** (framework overhead dominates):
| Topology | KPN++ | TBB |
|---|---|---|
| chain depth-1 | 1.7 | **1.4** |
| chain depth-4 | 2.5 | **2.2** |
| chain depth-8 | **3.0** | 3.6 |
| chain depth-16 | **9.3** | 13.0 |
| chain depth-32 | 23.2 | **14.2** |
| wide fanout-4 | 2.5 | **1.4** |
| diamond (2×2) | 3.4 | **1.9** |
Overhead µs/item at **work_us = 100** (moderate compute, KPN wins):
| Topology | KPN++ | TBB |
|---|---|---|
| chain depth-1 | **2.1** | 3.5 |
| chain depth-4 | **4.3** | 5.2 |
| chain depth-8 | **6.7** | 8.5 |
| chain depth-16 | **12.8** | 17.4 |
| chain depth-32 | **77** | 81 |
| wide fanout-4 | 3.4 | **1.9** |
| diamond (2×2) | **4.1** | 6.1 |
KPN++ pools beat TBB for every chain and diamond topology at 100 µs/node, and
match TBB within ~20% at 10 µs/node for shallow chains. TBB retains an edge on wide
fanout (serial dispatch loop vs. work-stealing pool) and at extreme oversubscription
depths (chain-32 at 10 µs). The remaining gap at light work is the cost of
`atomic::wait` vs. TBB's continuously-spinning worker threads.
### vs. TBB — API
The function signature is the node. KPN infers input and output types automatically;
there is no graph object to manage.
**Single-output node:**
```cpp
// KPN — 1 line
int scale(int x) { return x * 2; }
// TBB — must state types, concurrency policy, and carry a graph reference
tbb::flow::function_node<int,int> n(g, tbb::flow::serial, [](int x){ return x*2; });
```
**Multi-output node:**
```cpp
// KPN — return a tuple
std::tuple<cv::Mat,cv::Mat> split(cv::Mat f) { return {f, f}; }
// TBB — multifunction_node + explicit try_put per port
tbb::flow::multifunction_node<cv::Mat, std::tuple<cv::Mat,cv::Mat>> n(
g, tbb::flow::serial,
[](cv::Mat f, auto& ports) {
std::get<0>(ports).try_put(f);
std::get<1>(ports).try_put(f);
});
```
**Named ports** — compile-time checked, zero runtime cost, not available in TBB:
```cpp
auto node = make_node<split>(in<"frame">{}, out<"colour","grey">{}, 5);
net.connect("cam", cam.output<"frame">(), "split", node.input<"frame">());
// ^^^^^^^ typo → compile error
```
| | KPN | TBB |
|---|---|---|
| Node definition | plain function | `function_node<In,Out>` + explicit types |
| Multi-output | `return std::tuple<A,B>` | `multifunction_node` + `try_put` × N |
| Named ports | `in<"name">` / `out<"name">` compile-time | none |
| Graph lifetime | none | `graph g` must outlive all nodes |
| Shutdown | `net.stop()` | `g.wait_for_all()` + manual |
| Python bindings | designed-in | none |
Build the benchmarks with:
```bash
cmake -B build -DKPN_BUILD_BENCHMARKS=ON
cmake --build build --target bench_pipeline
./build/benchmarks/bench_pipeline | tee results.csv
```
---
## Project Structure
```
include/kpn/
fixed_string.hpp — NTTP string, in<>/out<> tags, index_of
traits.hpp — function_traits, normalised_return_t, output_count_v
channel.hpp — Channel<T>, channel_storage_policy, exceptions
port.hpp — InputPort<N,I>, OutputPort<N,I>
node.hpp — Node<Func,in<...>,out<...>>, make_node, INode
network.hpp — Network (builder, cycle detection, watchdog)
variant_node.hpp — VariantNode, PythonConverter<T>, unique_types (Python layer)
python/
bindings.hpp — nanobind helpers, GIL rule documentation
kpn.hpp — umbrella header
src/
network.cpp — non-template Network implementation
tests/
test_fixed_string.cpp
test_traits.cpp
test_channel.cpp
test_node.cpp
test_network.cpp
python/
kpn_python.cpp — nanobind module entry point
examples/
01_hello_pipeline/ … 09_opencv_cellshade/
scripts/
render_readme.py — regenerates README.md from README.md.in
```
---
## Contributing
Contributions are welcome. This project is hosted on a self-hosted Gitea
instance that accepts sign-in and registration with a GitHub account, so you
can log in with your existing GitHub identity to open issues and pull requests.
If you change any code that appears in a README snippet, edit `README.md.in`
(the template) rather than `README.md` directly, then regenerate:
```bash
cmake --build build --target readme # or: python scripts/render_readme.py
```
---
## Acknowledgments
AI tooling was used heavily throughout the development of this project,
including the design, implementation, tests, and documentation. All output
has been reviewed, but please keep this in mind when reading or building on the
code.
---
## License
Released under the [MIT License](LICENSE). Copyright (c) 2026 Duncan Tourolle.
+789
View File
@@ -0,0 +1,789 @@
# KPN++ — Kahn Process Network Library Specification
## Overview
A header-only C++20 template-metaprogramming library for building Kahn Process Networks. Each
node wraps a function (or callable object); its input types are inferred from the parameter
list and its output types from the return type. Nodes communicate over bounded, lock-free
SPSC FIFO channels.
Unlike a naive "one blocking thread per node" model, KPN++ is **reactive**: a node is
scheduled onto a thread pool whenever all of its input channels have data. A node that wraps a
function with `Node<>` owns a private single-thread pool and behaves exactly like an
independent worker; multiple nodes can instead share one `ThreadPool` for bounded-thread
execution. Source nodes self-resubmit; event-driven sources (`InterruptNode`) fire on an
external trigger.
The library ships rich runtime diagnostics (per-node exec/CPU/throughput stats, per-channel
fill/bandwidth/overflow counters, pool and shared-resource utilisation), an optional in-process
web debug UI, and nanobind-based Python bindings (partially implemented).
> **Note on accuracy.** This document describes the code as it exists in `include/kpn/`. Where
> a behaviour is subtle the relevant header is named so the source remains the ground truth.
---
## Project Structure
```
kpn++/
├── CMakeLists.txt
├── include/kpn/
│ ├── fixed_string.hpp # NTTP string + in<>/out<> tags + index_of
│ ├── traits.hpp # function signature introspection, normalised_return_t, repeat_tuple
│ ├── diagnostics.hpp # NodeStats, ChannelStats, *Snapshot, IPoolProbe, IResourceProbe
│ ├── channel.hpp # lock-free SPSC ring-buffer Channel<T> + storage policy
│ ├── port.hpp # InputPort / OutputPort handles
│ ├── inode.hpp # INode interface, NodeErrorHandler, NodeEvent
│ ├── scheduler.hpp # IScheduler + work-stealing ThreadPool
│ ├── pool_node.hpp # PoolNode / PoolObjectNode (reactive, scheduler-driven)
│ ├── interrupt_node.hpp # InterruptNode (external-trigger source)
│ ├── node.hpp # Node / ObjectNode (PoolNode + private 1-thread pool) + make_node
│ ├── fanout.hpp # FanoutNode<T,N> + make_fanout
│ ├── branch.hpp # RouterNode<T,N> + FilterNode<T> + make_router / make_filter
│ ├── shared_resource.hpp # SharedResource<T> priority-arbitrated exclusive resource
│ ├── main_thread_node.hpp # MainThreadNode<> (GUI / main-thread-bound nodes)
│ ├── static_network.hpp # Edge<>, make_network(), StaticNetwork<>
│ ├── network.hpp # runtime Network builder + watchdog + diagnostics
│ ├── debug_hub.hpp # DebugHub multi-network web UI (KPN_WEB_DEBUG only)
│ ├── web_debug.hpp # single-network web debug server (KPN_WEB_DEBUG only)
│ ├── variant_node.hpp # runtime-typed nodes/channels for Python graphs
│ ├── tmp/
│ │ ├── fanout_groups.hpp # compile-time fan-out detection + edge expansion
│ │ ├── topo_sort.hpp # compile-time DFS cycle check + topological order
│ │ └── repeat_tuple.hpp # repeat_tuple_t<T,N>
│ ├── python/
│ │ ├── bindings.hpp # PyNetwork / PyNode nanobind helpers
│ │ └── auto_bind.hpp # NodeRegistry / Entry / bind_network / bind_debug
│ └── kpn.hpp # umbrella header
├── src/network.cpp
├── tests/ # Catch2 v3 + GoogleTest
├── examples/ # 0116 (see Examples)
├── benchmarks/ # bench_pipeline (optional, KPN_BUILD_BENCHMARKS)
└── python/kpn_python.cpp # nanobind module definition
```
`kpn.hpp` is the umbrella header; including it pulls in the full C++ API (the Python layer is
included only by the binding TU).
---
## Component 0 — `fixed_string.hpp`: NTTP String + Port Tags
Named ports use C++20 non-type template parameters (NTTPs). `std::string_view` and
`const char*` are not valid NTTPs, so a `fixed_string` literal type provides `constexpr`
internal storage.
```cpp
template<std::size_t N>
struct fixed_string {
char data[N]{};
constexpr fixed_string(const char (&s)[N]) { std::copy_n(s, N, data); }
constexpr bool operator==(const fixed_string&) const = default;
constexpr std::string_view view() const { return {data, N - 1}; }
};
template<std::size_t N>
fixed_string(const char (&)[N]) -> fixed_string<N>; // deduction guide (required)
```
`fixed_string<4>` and `fixed_string<7>` are distinct types, so `input<"img">()` and
`input<"sigma">()` produce different instantiations — enabling zero-overhead compile-time
port dispatch.
Named-port lookup uses a `constexpr` `index_of` over the name pack; it returns the sentinel
`npos` on a miss so the `static_assert` fires at the `input<"img">()` **call site**, giving a
readable error at the point of use:
```cpp
inline constexpr std::size_t npos = std::size_t(-1);
template<fixed_string Name, fixed_string... Names>
constexpr std::size_t index_of(); // returns position or npos
```
### Port tags
`in<...>` and `out<...>` tag types disambiguate input vs. output name packs in the factory
API. Both are trivial empty structs; both are optional (omit to get index-only ports).
```cpp
template<fixed_string... Names> struct in {};
template<fixed_string... Names> struct out {};
```
> There is **no `latch<>` tag.** An earlier design sketched latched (most-recent-value)
> input ports; this was not implemented and the only input kind is the synchronous one.
---
## Component 1 — `traits.hpp`: Function Introspection
Extracts parameter and return types from any callable at compile time, for free functions,
function pointers, member function pointers (const and non-const), lambdas and `std::function`.
```cpp
// function_traits<F>::return_t, ::args (std::tuple<...>), ::arity
template<typename F> using return_t = ...; // return type
template<typename F> using args_t = ...; // std::tuple of parameters
template<typename F> inline constexpr std::size_t arity_v = ...;
```
The return type is normalised to a tuple so every node has a uniform output-tuple shape:
```cpp
// void → std::tuple<> (sink node, 0 outputs)
// T (non-tup) → std::tuple<T> (1 output)
// tuple<...> → tuple<...> (one output port per element)
template<typename T> using normalised_return_t = ...;
template<typename F> inline constexpr std::size_t output_count_v = ...;
```
`repeat_tuple_t<T, N>` (also surfaced via `tmp/repeat_tuple.hpp`) builds `std::tuple<T, …, T>`
with `N` repetitions — used by `FanoutNode` and `RouterNode` to describe their N identical
output ports.
---
## Component 2 — `diagnostics.hpp`: Statistics and Snapshots
Shared timing types: `clock_t = std::chrono::steady_clock`, `duration_t` is a
`double`-millisecond duration.
- **`NodeStats`** — atomic counters updated per fire: `frames_processed`, an EMA of wall-clock
exec time (`ema_exec_us`, warmup-mean for the first 5 frames then α=0.1), `max_exec_us`,
`total_blocked_us`, thread CPU time (`total_cpu_us` via `CLOCK_THREAD_CPUTIME_ID`),
`queue_wait_us` (pool queue latency), and `exec_start_us` (non-zero while executing; used by
the watchdog to detect hung nodes).
- **`ChannelStats`** — `pushes`, `bytes_pushed`, `drops`, `overflows`, `pops`, `peak_fill`.
- **Snapshots** — copyable plain structs taken by the watchdog / UI: `NodeSnapshot`,
`ChannelSnapshot` (with `fill_pct()`, `peak_pct()`, `bandwidth_mbs()`), `PoolSnapshot`,
`ResourceSnapshot`, and `NetworkSnapshot` (used by the `DebugHub`).
- **Probe interfaces** — `IPoolProbe` and `IResourceProbe` expose a `snapshot(name)` method so
pools and shared resources can be registered with a network for reporting.
### `ChannelDataSize<T>` trait
`bytes_pushed` is computed from a specialisable trait, defaulting to `sizeof(T)`. Specialise it
for heap-owning payloads to get accurate bandwidth:
```cpp
template<> struct kpn::ChannelDataSize<cv::Mat> {
static std::size_t bytes(const cv::Mat& m) { return m.total() * m.elemSize(); }
};
```
---
## Component 3 — `channel.hpp`: Lock-free Bounded FIFO + Storage Policy
### Storage policy
The type stored inside a channel is chosen by a specialisable trait. Small trivially-copyable
types are stored by value; everything else as `std::shared_ptr<const T>` so fan-out copies a
refcount, not data:
```cpp
template<typename T>
struct channel_storage_policy {
static constexpr bool by_value =
std::is_trivially_copyable_v<T> && sizeof(T) <= 8;
};
template<typename T>
using channel_storage_t = std::conditional_t<
channel_storage_policy<T>::by_value, T, std::shared_ptr<const T>>;
```
Override it to force value semantics for a custom small type. Push wraps a value in
`make_shared<const T>` when needed; pop dereferences it transparently, so a function taking
`const T&` works naturally and immutability is compiler-enforced.
### Channel — SPSC ring buffer
`Channel<T>` is a single-producer/single-consumer ring buffer (capacity rounded up to a power
of two). It uses C++20 `std::atomic::wait/notify_one` (portable futex) with a configurable
**spin-before-sleep** window so the common case never touches the kernel.
```cpp
template<typename T>
class Channel {
public:
using storage_type = channel_storage_t<T>;
explicit Channel(std::size_t capacity = 5, std::size_t spin_count = 200);
void push(T value); // drops if disabled; throws ChannelOverflowError if full
bool push_sentinel(T value); // out-of-band, non-blocking must-deliver token (EOF)
T pop(); // blocks (spin then futex); throws ChannelClosedError if disabled+empty
bool try_pop(T& out, std::chrono::milliseconds timeout); // polling (watchdog/display)
bool try_pop_now(T& out); // immediate, non-blocking
void enable(); // accept pushes
void disable(); // stop accepting + unblock any waiting pop()
void set_push_callback(std::function<void()>); // empty→non-empty notification
std::size_t size() const; // ring occupancy (excludes any pending sentinel)
std::size_t approx_size() const; // size() + 1 if a sentinel is pending (readiness checks)
std::size_t capacity() const;
bool is_accepting() const;
const ChannelStats& stats() const;
ChannelSnapshot snapshot(const std::string& name) const;
};
class ChannelOverflowError : public std::runtime_error { /* capacity + optional context */ };
class ChannelClosedError : public std::runtime_error {};
```
`head_` and `tail_`/`wake_` live on separate cache lines (`alignas(64)`) to avoid false
sharing between producer and consumer. `spin_hint()` issues a `pause`/`yield` instruction (or a
compiler fence on other ISAs).
### The `push_callback` — how reactivity works
`set_push_callback` registers a callback fired when a channel transitions empty→non-empty. A
consuming `PoolNode` installs this on each of its input channels; when an input becomes ready it
re-evaluates whether **all** inputs have data and, if so, submits itself to the scheduler. This
is the mechanism that replaces a dedicated blocking thread per node.
### The out-of-band EOF sentinel — `push_sentinel`
`push_sentinel(T value)` delivers a **must-deliver control token** (a graceful-EOF marker) that
cannot be dropped by backpressure. The value is stored in a dedicated slot **outside** the ring,
so it consumes no capacity, never throws `ChannelOverflowError`, and never blocks the producer.
This matters because a node's worker cannot afford to block on a downstream push: parking that
thread would stop it draining its own input, cascading into a hold-and-wait deadlock under
backpressure. `push_sentinel` sets a published flag (`has_eof_`) and returns immediately, keeping
the worker free to keep popping.
Ordering is preserved: the consumer's `pop()` / `try_pop_now()` drain the ring **first** and only
surface the sentinel once the ring is observed empty — so EOF always arrives after every value
pushed before it. `approx_size()` (used by node readiness checks) counts a pending sentinel as one
consumable item, so a channel carrying *only* a sentinel still schedules its consumer's next fire
and the token is never stranded. Same SPSC contract as `push()` (sole producer); returns `false`
if the channel is already disabled (teardown in progress → the token is moot).
### Backpressure and shutdown — `accepting_` flag
Each channel carries `std::atomic<bool> accepting_` (default `true`). It is the primary shutdown
mechanism; the only additional signal is the out-of-band EOF sentinel above, used for *graceful*
drain rather than an abrupt close.
- **`push()`** on a disabled channel silently drops the value (recorded as a `drop`). On a
full accepting channel it throws `ChannelOverflowError` (a sizing error).
- **`pop()`** blocks while empty and accepting; `disable()` wakes it and it throws
`ChannelClosedError`.
The **consumer node** owns its input channels and flips the flag: `start()` calls `enable()`,
`stop()` calls `disable()`. Producers never touch it.
### Ownership
Input channels are owned by their **consumer node** (held as `shared_ptr<Channel<T>>`). A
producer node holds a non-owning raw `Channel<T>*` to push into. `Network`/`StaticNetwork` are
otherwise non-owning of user nodes — see Components 89.
---
## Component 4 — `inode.hpp`: The Node Interface
Every node implements `INode`:
```cpp
struct INode {
virtual ~INode() = default;
virtual void start() = 0;
virtual void stop() = 0;
virtual bool running() const = 0;
virtual const NodeStats& stats() const = 0;
virtual NodeSnapshot node_snapshot(const std::string& name, double elapsed_s) const = 0;
virtual void set_name(std::string) = 0;
virtual void set_network_overflow_callback(NodeEventCallback) {} // network-injected
virtual void set_network_closed_callback(NodeEventCallback) {}
virtual void halt() { stop(); } // immediate, discard in-flight work
virtual void shutdown() { stop(); } // graceful topo-ordered drain (overridden by networks)
};
```
Supporting types:
```cpp
// Per-node error policy: return true to skip the failed fire and keep running,
// false to stop the node (and signal closed downstream).
using NodeErrorHandler = std::function<bool(std::string_view node_name, std::exception_ptr)>;
using NodeEventCallback = std::function<void(std::chrono::steady_clock::time_point)>;
enum class NodeEvent { Overflow, Closed };
```
---
## Component 5 — `scheduler.hpp`: Thread Pool
```cpp
struct IScheduler {
virtual void submit(std::function<void()> task, float priority = 0.5f) = 0;
virtual void start() = 0;
virtual void stop() = 0; // join workers, discard pending tasks
virtual void drain() = 0; // block until in-flight tasks complete (workers keep running)
};
```
`ThreadPool` is a **work-stealing** pool implementing both `IScheduler` and `IPoolProbe`. Each
worker owns a priority queue (max-heap by `priority`, FIFO within equal priority via a sequence
counter). `submit()` distributes round-robin; idle workers steal from the most-loaded peer
using `try_lock`, then sleep on a shared condition variable. The submit/notify path takes the CV
mutex around `notify` to close the lost-wakeup window; `drain()` waits on a separate counter of
in-flight tasks. `priority` lets a hot node (full input, empty output) be scheduled ahead of
others — see `PoolNode::compute_priority`.
---
## Component 6 — Node Types
All processing nodes share the same shape: typed input channels they own, raw output-channel
pointers set at wiring time, `args_tuple` / `return_tuple` aliases used by the connect-time
type check, and `static constexpr` `label()` / `unique_tag` / `input_count` / `output_count`.
### `PoolNode` / `PoolObjectNode` — reactive, scheduler-driven (`pool_node.hpp`)
The core node. Instead of a blocked thread, it submits a `fire_once()` to a shared
`IScheduler` whenever all inputs are ready; `queued_` ensures at most one `fire_once()` is
in flight. `fire_once()` pops every input (`try_pop_now`), runs the function, pushes each
normalised output, records stats, then resubmits if inputs remain ready. Source nodes
(`input_count == 0`) self-submit on `start()` and after each fire.
```cpp
template<auto Func,
typename InputTag = in<>,
typename OutputTag = out<>,
fixed_string Label = "",
std::size_t UniqueTag = 0>
class PoolNode : public INode { ... };
auto n = make_pool_node<func>(scheduler, fifo_capacity); // index ports
auto n = make_pool_node<func, "label", 0>(scheduler, in<"a">{}, out<"b">{}, cap);
```
`PoolObjectNode<Obj, …>` is the same for a stateful callable object (introspected via
`&Obj::operator()`); the object must outlive the node.
Per-node configuration: `set_error_handler(NodeErrorHandler)`, `set_overflow_callback`,
`set_closed_callback`, `set_max_exec_time`. Inside `fire_once()`:
`ChannelOverflowError` fires the overflow callbacks; `ChannelClosedError` (or an error handler
returning `false`) fires the closed callbacks and self-stops; any other exception consults the
error handler.
**Name-count contract** — a `static_assert` requires that the number of input names is `0` or
equals arity (same for outputs):
```cpp
static_assert(sizeof...(InNames) == 0 || sizeof...(InNames) == input_count,
"make_pool_node: number of input names must match function arity, or provide none");
```
### `Node` / `ObjectNode` — convenience wrappers (`node.hpp`)
`Node<>` privately owns a `ThreadPool(1)` and derives from `PoolNode<>` with the **same**
template signature, so each `Node` is a self-contained worker with no external scheduler. Its
`start()`/`stop()` start and stop the private pool around the base. This keeps the simple API —
`make_node<func>(5)` — while routing all execution through the one `fire_once()` code path.
```cpp
template<auto Func, typename InputTag = in<>, typename OutputTag = out<>,
fixed_string Label = "", std::size_t UniqueTag = 0>
class Node : public PoolNode<...> { ... };
auto src = make_node<produce>(5);
auto dbl = make_node<double_it, "dbl">(5);
auto cnt = make_node<count_words>(in<"words">{}, out<"count","words">{}, 4);
```
The `Label` NTTP gives a human-readable name for diagnostics; `UniqueTag` is a collision-breaker
required when the **same function** is used as two distinct vertices in a `StaticNetwork` (two
`make_node<blur>` would otherwise be the same type). Both default so existing code is unaffected.
To share one pool across many nodes for bounded-thread execution, use `make_pool_node` directly.
### `InterruptNode` — external-trigger source (`interrupt_node.hpp`)
A zero-input source driven by an external event (camera frame, timer, socket) instead of
self-resubmission. `get_trigger()` returns a thread-safe callable to hand to the event source;
each call increments a `pending_` counter and submits `fire_once()` on the 0→1 transition,
guaranteeing one execution per trigger even under bursts. It does not busy-loop.
```cpp
auto cam = make_interrupt_node<grab_frame>(scheduler, out<"frame">{});
camera_sdk.on_frame_ready(cam.get_trigger());
```
### `FanoutNode<T, N>` — explicit fan-out (`fanout.hpp`)
Reads one item and pushes a copy to each of N outputs (per-output overflow drops
independently). Runs on its own `std::jthread` blocking on `pop()`. Used directly in a runtime
`Network` via `make_fanout<T,N>`, and auto-inserted by `make_network()` for `StaticNetwork`.
### `RouterNode<T, N>` / `FilterNode<T>` — branching (`branch.hpp`)
Both run on a dedicated `jthread`. `RouterNode` pushes each item to exactly **one** of N
outputs chosen by a `selector(item) -> size_t` (out-of-range index drops). `FilterNode` forwards
an item only when `pred(item)` is true. Factories: `make_router<T,N>(sel)`, `make_filter<T>(pred)`.
### `MainThreadNode<Derived, in<…>, Args…>` — GUI / main-thread nodes (`main_thread_node.hpp`)
For work that *must* run on the thread owning a GUI event loop (OpenCV `imshow`/`waitKey` on
Wayland/Qt). It owns input channels and is registered as a normal `INode` (appears in
diagnostics) but spawns **no** thread. The application drives it by calling `step()` in a loop on
the main thread: `step()` does a zero-timeout `try_pop` on every input, and when all are ready
invokes the derived `operator()(Args…)` (returning `false` to stop). CRTP; the derived class
supplies the operator.
---
## Component 7 — `shared_resource.hpp`: Priority-arbitrated Exclusive Resource
`SharedResource<T>` wraps a singleton-like resource (an ONNX session, a CUDA stream) shared by
nodes across one or more networks, and arbitrates access with a **priority + aging** waiter
queue. Priority is re-evaluated at every release (so it reflects current queue state), and each
waiter's effective score grows with wait time (`kAgingPerSecond`) to prevent starvation.
```cpp
SharedResource<OrtSession> res(session_args...);
// inside a node functor:
auto guard = res.acquire_balanced(in_channel, out_channel); // RAII; releases on scope exit
guard->Run(...);
```
`acquire_balanced(in, out)` scores a waiter by `input_fill × output_headroom` — a node with a
full input queue and empty output is most urgent. `acquire(fn)` takes any `()->float` priority;
`acquire()` treats all waiters equally. Implements `IResourceProbe` so it shows up in
diagnostics and the debug hub. The factory is `make_shared_resource<T>(args…)`.
---
## Component 8 — `network.hpp`: Runtime Graph Builder + Watchdog
`Network` is **non-owning** (nodes outlive it; `add()` stores `INode*`). A builder collects the
full topology before `build()`, enabling cycle detection and topological ordering.
```cpp
class Network : public INode {
public:
template<typename NodeT> Network& add(std::string name, NodeT& node);
template<typename SrcNode, std::size_t SrcIdx, typename DstNode, std::size_t DstIdx>
Network& connect(const std::string& src, OutputPort<SrcNode, SrcIdx>,
const std::string& dst, InputPort<DstNode, DstIdx>);
Network& expose_input (std::string boundary_name, InputPort<NodeT, Idx>); // sub-network port
Network& expose_output(std::string boundary_name, OutputPort<NodeT, Idx>);
Network& build(); // DFS cycle check (throws NetworkCycleError) + topo sort
void start() override; // start nodes in topo order; launch watchdog (+ web UI)
void stop() override; // == halt()
void halt() override; // immediate: stop nodes in reverse topo order
void shutdown() override; // graceful: stop source layers, drain channels, descend
void set_watchdog_interval(std::chrono::milliseconds);
void set_error_handler(ErrorHandler); // void(node_name, exception_ptr)
void set_diagnostics_handler(DiagnosticsHandler); // fired each watchdog tick
void set_event_handler(EventHandler); // void(name, NodeEvent, timestamp)
void register_pool(const std::string&, IPoolProbe*);
void print_diagnostics(std::ostream& = std::cerr) const; // formatted table
};
```
- **`connect`** static-asserts that the source output type equals the destination input type
(via the nodes' `return_tuple` / `args_tuple`), sets the consumer's input channel as the
producer's output pointer, registers a `ChannelProbe` for diagnostics, and rejects a second
connection from the same output port (use `make_fanout`).
- **`build`** colours the graph DFS; a back-edge throws `NetworkCycleError`. It also wires each
node's network-level overflow/closed callbacks to the `EventHandler` if one is set.
- **`halt` vs `shutdown`** — `halt()` disables channels and stops nodes in reverse order
immediately; `shutdown()` walks source layers first, polling channel probes until they drain
before stopping the next layer.
- **Watchdog** — a `std::jthread` that wakes on `watchdog_interval_` (default 3 s), collects
snapshots, warns about nodes whose `exec_start_us` indicates an execution running > 5 s, and
either calls the diagnostics handler or prints the formatted report.
- **`expose_input`/`expose_output`** record boundary names (sub-network support is scaffolded;
`Network` is itself an `INode` and can be `add()`ed to an outer `Network`).
The formatted report includes node (frames, exec ms, max ms, blocked ms, fps, cpu ms, util%),
channel (fill%, peak%, pushes, drops, overflow, MB/s, item bytes), and pool tables, plus a
bottleneck hint (highest `ema_exec_ms`).
---
## Component 9 — `static_network.hpp`: Compile-time Graph Builder
For C++ graphs whose full topology is known at compile time. The complete edge list is a type
pack, so fan-out arity is known up front, cycle detection is a `static_assert`, and start/stop
are pointer-vector traversals rather than string-map + virtual dispatch.
```cpp
// edge() builds a typed Edge descriptor from two port handles.
template<typename SrcNode, std::size_t SrcIdx, typename DstNode, std::size_t DstIdx>
Edge<SrcNode, SrcIdx, DstNode, DstIdx>
edge(OutputPort<SrcNode, SrcIdx>, InputPort<DstNode, DstIdx>);
// make_network() takes all edges, expands fan-outs, wires channels, returns a StaticNetwork.
template<typename... Edges> auto make_network(Edges&&... edges);
```
Usage — no `add`/`connect`/`build`/string names; one source port feeding two destinations
auto-inserts a `FanoutNode`:
```cpp
auto src = make_node<produce, "src">(8);
auto blur = make_node<blur_func, "blur">(8);
auto detect = make_node<detect_func,"detect">(8);
auto sink = make_node<display, "sink">(8);
auto net = make_network(
edge(src.output<0>(), blur.input<0>()),
edge(src.output<0>(), detect.input<0>()), // same source port → FanoutNode<T,2> inserted
edge(blur.output<0>(), sink.input<0>()),
edge(detect.output<0>(), sink.input<1>()));
net.start(); /* … */ net.stop();
```
`make_network` performs, at compile time: fan-out detection and edge expansion
(`tmp/fanout_groups.hpp`), a duplicate-`(Func, UniqueTag)` check
(`static_assert` — "add a UniqueTag"), and a cycle check + topological order
(`tmp/topo_sort.hpp`, `static_assert` — "graph contains a directed cycle"). At run time it
heap-allocates owned `FanoutNode` storage, collects user-node pointers in edge order, sets each
node's display name (`Label`, else `node[UniqueTag]`; fan-outs become `"<src>_fanout"`), wires
every expanded edge, and builds channel probes.
`StaticNetwork<FanoutStorage, TopoNodeList>` implements `INode` (so it can be embedded in a
runtime `Network`). It owns the fan-out nodes, holds user nodes by pointer, and provides
`start`/`halt`/`shutdown`, an `EventHandler`, `register_resource` / `register_pool`,
`print_diagnostics`, and `network_snapshot()` (consumed by the `DebugHub`). Compile-time labels
are read from each `NodeType::label()`.
> `make_fanout<T,N>` remains for explicit fan-out in a runtime `Network`; `make_network` users
> never call it.
---
## Component 10 — Web Debugging (optional, `KPN_WEB_DEBUG`)
Zero cost when disabled — guarded headers, no symbols, no dependency. Depends on **cpp-httplib**
(single-header, fetched by CMake when the option is on) and loads **D3.js v7** from CDN. Enable
per-target:
```cpp
#define KPN_WEB_DEBUG 1
#include <kpn/kpn.hpp>
```
### Single-network server (`web_debug.hpp`)
When enabled, `Network` / `StaticNetwork` gain `set_web_debug_port(uint16_t)` (default 9090) and
auto-start an in-process HTTP server in `start()`. It serves an inline single-page app at `/` and
a JSON snapshot at `/api/snapshot` (nodes, channels/edges, pools, resources, elapsed). The page
renders a force-directed graph: node colour encodes `ema_exec_ms`, edge colour encodes fill%,
with hover tooltips for the full stat set; it polls every 500 ms.
### `DebugHub` — multi-network UI (`debug_hub.hpp`)
A standalone server aggregating several networks under one endpoint:
```cpp
DebugHub hub(9090);
hub.register_network("detect", detect_net); // disables that net's own server
hub.register_network("classify", classify_net);
hub.register_resource("gpu", &gpu_resource); // shows utilisation cards
hub.start();
```
The hub UI has one tab per registered network plus an "All Networks" tab with shared-resource
cards and a cross-network node table. `register_network` calls `net.disable_web_server()` so the
hub is the single debug endpoint; call it before `net.start()`.
---
## Component 11 — Python Bindings (partial)
> Status: scaffolded and partially implemented. The variant machinery, `PyNetwork`/`PyNode`, and
> the auto-binding layer exist; the demo module wires a hello-pipeline. Full sub-port read/write
> and mixed C++/Python graphs are still in progress.
Python graphs cannot resolve types at compile time, so a `PyNetwork` is parameterised by a
`std::variant` derived (at compile time, via `unique_types`) from the port types of a **closed
list of registered C++ node types**. The variant only appears at the C++/Python boundary; each
node's internal `Channel<T>` still stores raw `T` (`variant_node.hpp`: `IVariantChannel`,
`VariantChannel<T,Variant>`, `IVariantNode`, `VariantNodeWrapper`).
### Auto-binding (`python/auto_bind.hpp`)
The node list is declared once with a `NodeRegistry` of `Entry<func, "name">`. `bind_network`
registers the `PyNetwork` class, a `make_<name>(capacity)` factory and a `<Name>Node` class per
entry, and auto-registers `PythonConverter` for each port type. `bind_debug` additionally exposes
each raw C++ function as a free Python callable for testing without a network. Recompiling the
extension is the registration step — there is no CMake code-gen.
```cpp
using DemoNodes = kpn::python::NodeRegistry<
kpn::python::Entry<produce, "produce">,
kpn::python::Entry<double_it, "double_it">,
kpn::python::Entry<print_it, "print_it">>; // variant auto-deduced as std::variant<int>
NB_MODULE(kpn_python, m) {
bind_network<DemoNodes>(m);
bind_debug<DemoNodes>(m);
}
```
Custom types are supported by specialising `kpn::PythonConverter<T>` (`to_python` / `from_python`,
optional `type_name`) before `bind_network`.
### GIL rules (non-negotiable)
1. **Acquire for callback** — hold the GIL only for the duration of a Python callable
invocation (`nb::gil_scoped_acquire` around the call site).
2. **Release while blocking** — release the GIL before any blocking channel op
(`nb::gil_scoped_release`), then re-acquire. Violating this deadlocks: a PyNode thread
waiting for the GIL cannot proceed while another thread holds it and blocks on a channel
waiting for that PyNode.
---
## Error Handling Contract
| Situation | Behaviour |
|---|---|
| FIFO overflow (full, accepting) | `ChannelOverflowError` thrown in producer; node overflow callbacks fire |
| Push to a disabled channel | Value silently dropped (counted as a `drop`) |
| Node function throws | Routed to the node's `NodeErrorHandler``true` skips & continues, `false` stops the node |
| Node stopped / channel closed | `ChannelClosedError` → node fires closed callbacks and self-stops |
| Type mismatch (C++) | `static_assert` at `connect()` / `make_network()` |
| Cycle in graph (runtime) | `NetworkCycleError` thrown at `build()` |
| Cycle in graph (static) | `static_assert` at `make_network()` |
| Duplicate `(Func, UniqueTag)` (static) | `static_assert` at `make_network()` — add a `UniqueTag` |
| Hung node | Watchdog warning after threshold |
`Network` additionally exposes an aggregate `EventHandler(name, NodeEvent, timestamp)` for
overflow/closed events across all nodes.
---
## Thread Model
KPN++ is **reactive**, not one-thread-per-node:
- A `PoolNode` owns no thread. It registers a push-callback on each input channel; when all
inputs are ready it submits `fire_once()` to a shared `IScheduler` (a `ThreadPool`).
- `Node<>` wraps a `PoolNode` plus a **private `ThreadPool(1)`**, recovering "independent
worker" semantics with the simple `make_node` API. Many nodes can instead share one pool
(`make_pool_node`) for a bounded OS thread count.
- `FanoutNode`, `RouterNode`, and `FilterNode` do run a dedicated `std::jthread` blocking on
`pop()` (they are simple, latency-sensitive routers).
- `InterruptNode` fires on an external trigger; `MainThreadNode` runs on the caller's main
thread via `step()`.
`std::jthread` (C++20) and its `stop_token` are used where a thread is owned, simplifying
cooperative shutdown. Benchmarks (`benchmarks/bench_pipeline`) show ~27 µs/hop framework
overhead for chains within the core count, rising under oversubscription.
---
## Platform and Compiler Requirements
C++20 is required.
| Feature | Min compiler |
|---|---|
| NTTP structural types (`fixed_string`) | GCC 11, Clang 13, MSVC 19.29 |
| `std::atomic::wait/notify` (channel futex) | GCC 11, Clang 13, MSVC 19.29 |
| `std::jthread` + `stop_token` | GCC 11, Clang 14, MSVC 19.29 |
| `auto` NTTPs, fold expressions, `if constexpr`, concepts | C++20 / C++17 baseline |
`CLOCK_THREAD_CPUTIME_ID` (per-thread CPU stats in `diagnostics.hpp`) is POSIX. nanobind
requires Python 3.8+ (auto-fetched when `KPN_BUILD_PYTHON=ON`).
---
## Testing Strategy
**Catch2 v3** for behaviour/integration tests and **GoogleTest** for unit and death tests; both
are auto-fetched. Existing suites: `test_fixed_string`, `test_traits`, `test_channel`,
`test_node`, `test_network`, `test_static_network`, `test_scheduler`, `test_pool_node`,
`test_shared_resource`.
Cases covered explicitly include: channel blocking/unblocking and overflow; shutdown races
(`stop()` while blocked on `pop()`); `try_pop_now`; fan-out delivery; tuple unpacking to
sub-channels; runtime cycle detection and static cycle/duplicate-tag `static_assert`s; named
port lookup and wrong-name-count `static_assert`s; storage-policy by-value vs `shared_ptr`;
scheduler submit/steal/drain; `PoolNode` reactive scheduling; and `SharedResource` priority +
aging.
---
## Examples
Self-contained programs under `examples/`, built by default (`-DKPN_BUILD_EXAMPLES=OFF` to
skip). They double as documentation and smoke tests.
| Example | What it shows |
|---|---|
| `01_hello_pipeline` | Linear pipeline, index-based wiring, `Network` builder |
| `02_named_ports` | `in<>`/`out<>` tags, named port access, wrong-name `static_assert` |
| `03_multi_output` | Tuple-returning node, per-element sub-port routing |
| `04_storage_policy` | `channel_storage_policy` default + specialisation |
| `05_error_handling` | `ChannelOverflowError`, diagnostics handler |
| `06_watchdog` | Watchdog interval, stall detection |
| `07_python_network` | `PyNetwork` with a pure-Python node *(pending)* |
| `08_python_subport` | `net.read` / `net.write`, sub-port tap *(pending)* |
| `09_opencv_cellshade` | Real-time cell-shading on webcam; named ports, fan-out, `MainThreadNode` display (requires OpenCV) |
| `10_static_hello_pipeline` | `make_network()` version of 01 — compile-time topology |
| `11_static_fanout` | Auto-inserted `FanoutNode` from a duplicated source port |
| `12_static_cellshade` | Static cell-shading with auto fan-out and `Label` NTTPs |
| `13_debug_cellshade` | One-op-per-node pipeline + variadic `DebugCanvas<N>` tiling node |
| `14_debug_hub` | Two networks sharing a `SharedResource` via `DebugHub` |
| `15_node_error_handler` | Per-node `set_error_handler` (skip-and-continue vs stop) |
| `16_event_callbacks` | `set_overflow_callback` + network `set_event_handler` |
---
## Future Extension Points (Heterogeneous Execution)
Not implemented, but the design keeps these doors open:
- **`IChannel` abstract interface** — `Channel<T>` and a future `RemoteChannel<T>` (socket /
shared-memory) sharing one `push`/`pop` surface so nodes are agnostic to channel location.
- **`Serializer<T>` trait** — parallel to `channel_storage_policy` / `PythonConverter`, for
cross-device serialisation (MessagePack for embedded, pinned memory for GPU zero-copy).
- **`NodeKind` tag** — e.g. `{ Local, Gpu, Remote }` on `INode`, letting the watchdog apply
per-device health-check and timeout strategies.
The `IScheduler` abstraction already decouples node execution from any specific thread model,
making a cooperative or device-specific executor a drop-in.
---
## Resolved Design Decisions
| Question | Decision |
|---|---|
| Execution model | Reactive: nodes submit `fire_once()` to an `IScheduler` when inputs are ready, not one blocking thread per node |
| `Node<>` vs `PoolNode<>` | `Node<>` owns a private `ThreadPool(1)`; `PoolNode<>` shares a pool for bounded threads |
| Channel | Lock-free SPSC ring buffer, `atomic::wait/notify` + spin-before-sleep |
| Shutdown | Per-channel `accepting_` flag; `disable()` unblocks `pop()` (→ `ChannelClosedError`) |
| Overflow | `ChannelOverflowError` on full accepting channel; silent drop on disabled channel |
| Node error policy | Per-node `NodeErrorHandler` returning bool (skip vs stop) |
| Network ownership | Non-owning; user declares nodes, network stores `INode*` |
| Fan-out | Explicit `FanoutNode<T,N>` for runtime `Network`; auto-inserted by `make_network()` |
| Branching | `RouterNode<T,N>` (select one of N) and `FilterNode<T>` (predicate gate) |
| Static vs runtime graph | Both; `StaticNetwork` for compile-time C++ topology, `Network` for dynamic/Python; `StaticNetwork` is an `INode` so it embeds in `Network` |
| Node identity (static graphs) | `Label` NTTP (name) + `UniqueTag` NTTP (collision-breaker); both default |
| Shared device resource | `SharedResource<T>` with priority + aging arbitration |
| Main-thread / GUI work | `MainThreadNode<>` driven by `step()` on the main thread |
| External-event sources | `InterruptNode` with a thread-safe `get_trigger()` |
| Web debugging | Per-network server + multi-network `DebugHub`, behind `KPN_WEB_DEBUG` |
| Mixed-rate latched inputs | **Not implemented** — no `latch<>` ports |
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.8 KiB

File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-18
View File
@@ -1,18 +0,0 @@
/*!
* Lunr languages, `Danish` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.da=function(){this.pipeline.reset(),this.pipeline.add(e.da.trimmer,e.da.stopWordFilter,e.da.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.da.stemmer))},e.da.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.da.trimmer=e.trimmerSupport.generateTrimmer(e.da.wordCharacters),e.Pipeline.registerFunction(e.da.trimmer,"trimmer-da"),e.da.stemmer=function(){var r=e.stemmerSupport.Among,i=e.stemmerSupport.SnowballProgram,n=new function(){function e(){var e,r=f.cursor+3;if(d=f.limit,0<=r&&r<=f.limit){for(a=r;;){if(e=f.cursor,f.in_grouping(w,97,248)){f.cursor=e;break}if(f.cursor=e,e>=f.limit)return;f.cursor++}for(;!f.out_grouping(w,97,248);){if(f.cursor>=f.limit)return;f.cursor++}d=f.cursor,d<a&&(d=a)}}function n(){var e,r;if(f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(c,32),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del();break;case 2:f.in_grouping_b(p,97,229)&&f.slice_del()}}function t(){var e,r=f.limit-f.cursor;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.find_among_b(l,4)?(f.bra=f.cursor,f.limit_backward=e,f.cursor=f.limit-r,f.cursor>f.limit_backward&&(f.cursor--,f.bra=f.cursor,f.slice_del())):f.limit_backward=e)}function s(){var e,r,i,n=f.limit-f.cursor;if(f.ket=f.cursor,f.eq_s_b(2,"st")&&(f.bra=f.cursor,f.eq_s_b(2,"ig")&&f.slice_del()),f.cursor=f.limit-n,f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(m,5),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del(),i=f.limit-f.cursor,t(),f.cursor=f.limit-i;break;case 2:f.slice_from("løs")}}function o(){var e;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.out_grouping_b(w,97,248)?(f.bra=f.cursor,u=f.slice_to(u),f.limit_backward=e,f.eq_v_b(u)&&f.slice_del()):f.limit_backward=e)}var a,d,u,c=[new r("hed",-1,1),new r("ethed",0,1),new r("ered",-1,1),new r("e",-1,1),new r("erede",3,1),new r("ende",3,1),new r("erende",5,1),new r("ene",3,1),new r("erne",3,1),new r("ere",3,1),new r("en",-1,1),new r("heden",10,1),new r("eren",10,1),new r("er",-1,1),new r("heder",13,1),new r("erer",13,1),new r("s",-1,2),new r("heds",16,1),new r("es",16,1),new r("endes",18,1),new r("erendes",19,1),new r("enes",18,1),new r("ernes",18,1),new r("eres",18,1),new r("ens",16,1),new r("hedens",24,1),new r("erens",24,1),new r("ers",16,1),new r("ets",16,1),new r("erets",28,1),new r("et",-1,1),new r("eret",30,1)],l=[new r("gd",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1)],m=[new r("ig",-1,1),new r("lig",0,1),new r("elig",1,1),new r("els",-1,1),new r("løst",-1,2)],w=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],p=[239,254,42,3,0,0,0,0,0,0,0,0,0,0,0,0,16],f=new i;this.setCurrent=function(e){f.setCurrent(e)},this.getCurrent=function(){return f.getCurrent()},this.stem=function(){var r=f.cursor;return e(),f.limit_backward=r,f.cursor=f.limit,n(),f.cursor=f.limit,t(),f.cursor=f.limit,s(),f.cursor=f.limit,o(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return n.setCurrent(e),n.stem(),n.getCurrent()}):(n.setCurrent(e),n.stem(),n.getCurrent())}}(),e.Pipeline.registerFunction(e.da.stemmer,"stemmer-da"),e.da.stopWordFilter=e.generateStopWordFilter("ad af alle alt anden at blev blive bliver da de dem den denne der deres det dette dig din disse dog du efter eller en end er et for fra ham han hans har havde have hende hendes her hos hun hvad hvis hvor i ikke ind jeg jer jo kunne man mange med meget men mig min mine mit mod ned noget nogle nu når og også om op os over på selv sig sin sine sit skal skulle som sådan thi til ud under var vi vil ville vor være været".split(" ")),e.Pipeline.registerFunction(e.da.stopWordFilter,"stopWordFilter-da")}});
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.hi=function(){this.pipeline.reset(),this.pipeline.add(e.hi.trimmer,e.hi.stopWordFilter,e.hi.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.hi.stemmer))},e.hi.wordCharacters="ऀ-ःऄ-एऐ-टठ-यर-िी-ॏॐ-य़ॠ-९॰-ॿa-zA-Z-zA-0-9-",e.hi.trimmer=e.trimmerSupport.generateTrimmer(e.hi.wordCharacters),e.Pipeline.registerFunction(e.hi.trimmer,"trimmer-hi"),e.hi.stopWordFilter=e.generateStopWordFilter("अत अपना अपनी अपने अभी अंदर आदि आप इत्यादि इन इनका इन्हीं इन्हें इन्हों इस इसका इसकी इसके इसमें इसी इसे उन उनका उनकी उनके उनको उन्हीं उन्हें उन्हों उस उसके उसी उसे एक एवं एस ऐसे और कई कर करता करते करना करने करें कहते कहा का काफ़ी कि कितना किन्हें किन्हों किया किर किस किसी किसे की कुछ कुल के को कोई कौन कौनसा गया घर जब जहाँ जा जितना जिन जिन्हें जिन्हों जिस जिसे जीधर जैसा जैसे जो तक तब तरह तिन तिन्हें तिन्हों तिस तिसे तो था थी थे दबारा दिया दुसरा दूसरे दो द्वारा न नके नहीं ना निहायत नीचे ने पर पहले पूरा पे फिर बनी बही बहुत बाद बाला बिलकुल भी भीतर मगर मानो मे में यदि यह यहाँ यही या यिह ये रखें रहा रहे ऱ्वासा लिए लिये लेकिन व वग़ैरह वर्ग वह वहाँ वहीं वाले वुह वे वो सकता सकते सबसे सभी साथ साबुत साभ सारा से सो संग ही हुआ हुई हुए है हैं हो होता होती होते होना होने".split(" ")),e.hi.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.hi.tokenizer=function(i){if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var t=i.toString().toLowerCase().replace(/^\s+/,"");return r.cut(t).split("|")},e.Pipeline.registerFunction(e.hi.stemmer,"stemmer-hi"),e.Pipeline.registerFunction(e.hi.stopWordFilter,"stopWordFilter-hi")}});
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.hy=function(){this.pipeline.reset(),this.pipeline.add(e.hy.trimmer,e.hy.stopWordFilter)},e.hy.wordCharacters="[A-Za-z԰-֏ff-ﭏ]",e.hy.trimmer=e.trimmerSupport.generateTrimmer(e.hy.wordCharacters),e.Pipeline.registerFunction(e.hy.trimmer,"trimmer-hy"),e.hy.stopWordFilter=e.generateStopWordFilter("դու և եք էիր էիք հետո նաև նրանք որը վրա է որ պիտի են այս մեջ ն իր ու ի այդ որոնք այն կամ էր մի ես համար այլ իսկ էին ենք հետ ին թ էինք մենք նրա նա դուք եմ էի ըստ որպես ում".split(" ")),e.Pipeline.registerFunction(e.hy.stopWordFilter,"stopWordFilter-hy"),e.hy.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}(),e.Pipeline.registerFunction(e.hy.stemmer,"stemmer-hy")}});
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.ja=function(){this.pipeline.reset(),this.pipeline.add(e.ja.trimmer,e.ja.stopWordFilter,e.ja.stemmer),r?this.tokenizer=e.ja.tokenizer:(e.tokenizer&&(e.tokenizer=e.ja.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.ja.tokenizer))};var t=new e.TinySegmenter;e.ja.tokenizer=function(i){var n,o,s,p,a,u,m,l,c,f;if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(t){return r?new e.Token(t.toLowerCase()):t.toLowerCase()});for(o=i.toString().toLowerCase().replace(/^\s+/,""),n=o.length-1;n>=0;n--)if(/\S/.test(o.charAt(n))){o=o.substring(0,n+1);break}for(a=[],s=o.length,c=0,l=0;c<=s;c++)if(u=o.charAt(c),m=c-l,u.match(/\s/)||c==s){if(m>0)for(p=t.segment(o.slice(l,c)).filter(function(e){return!!e}),f=l,n=0;n<p.length;n++)r?a.push(new e.Token(p[n],{position:[f,p[n].length],index:a.length})):a.push(p[n]),f+=p[n].length;l=c+1}return a},e.ja.stemmer=function(){return function(e){return e}}(),e.Pipeline.registerFunction(e.ja.stemmer,"stemmer-ja"),e.ja.wordCharacters="一二三四五六七八九十百千万億兆一-龠々〆ヵヶぁ-んァ-ヴーア-ン゙a-zA-Z-zA-0-9-",e.ja.trimmer=e.trimmerSupport.generateTrimmer(e.ja.wordCharacters),e.Pipeline.registerFunction(e.ja.trimmer,"trimmer-ja"),e.ja.stopWordFilter=e.generateStopWordFilter("これ それ あれ この その あの ここ そこ あそこ こちら どこ だれ なに なん 何 私 貴方 貴方方 我々 私達 あの人 あのかた 彼女 彼 です あります おります います は が の に を で え から まで より も どの と し それで しかし".split(" ")),e.Pipeline.registerFunction(e.ja.stopWordFilter,"stopWordFilter-ja"),e.jp=e.ja,e.Pipeline.registerFunction(e.jp.stemmer,"stemmer-jp"),e.Pipeline.registerFunction(e.jp.trimmer,"trimmer-jp"),e.Pipeline.registerFunction(e.jp.stopWordFilter,"stopWordFilter-jp")}});
-1
View File
@@ -1 +0,0 @@
module.exports=require("./lunr.ja");
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.kn=function(){this.pipeline.reset(),this.pipeline.add(e.kn.trimmer,e.kn.stopWordFilter,e.kn.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.kn.stemmer))},e.kn.wordCharacters="ಀ-಄ಅ-ಔಕ-ಹಾ-ೌ಼-ಽೕ-ೖೝ-ೞೠ-ೡೢ-ೣ೤೥೦-೯ೱ-ೳ",e.kn.trimmer=e.trimmerSupport.generateTrimmer(e.kn.wordCharacters),e.Pipeline.registerFunction(e.kn.trimmer,"trimmer-kn"),e.kn.stopWordFilter=e.generateStopWordFilter("ಮತ್ತು ಈ ಒಂದು ರಲ್ಲಿ ಹಾಗೂ ಎಂದು ಅಥವಾ ಇದು ರ ಅವರು ಎಂಬ ಮೇಲೆ ಅವರ ತನ್ನ ಆದರೆ ತಮ್ಮ ನಂತರ ಮೂಲಕ ಹೆಚ್ಚು ನ ಆ ಕೆಲವು ಅನೇಕ ಎರಡು ಹಾಗು ಪ್ರಮುಖ ಇದನ್ನು ಇದರ ಸುಮಾರು ಅದರ ಅದು ಮೊದಲ ಬಗ್ಗೆ ನಲ್ಲಿ ರಂದು ಇತರ ಅತ್ಯಂತ ಹೆಚ್ಚಿನ ಸಹ ಸಾಮಾನ್ಯವಾಗಿ ನೇ ಹಲವಾರು ಹೊಸ ದಿ ಕಡಿಮೆ ಯಾವುದೇ ಹೊಂದಿದೆ ದೊಡ್ಡ ಅನ್ನು ಇವರು ಪ್ರಕಾರ ಇದೆ ಮಾತ್ರ ಕೂಡ ಇಲ್ಲಿ ಎಲ್ಲಾ ವಿವಿಧ ಅದನ್ನು ಹಲವು ರಿಂದ ಕೇವಲ ದ ದಕ್ಷಿಣ ಗೆ ಅವನ ಅತಿ ನೆಯ ಬಹಳ ಕೆಲಸ ಎಲ್ಲ ಪ್ರತಿ ಇತ್ಯಾದಿ ಇವು ಬೇರೆ ಹೀಗೆ ನಡುವೆ ಇದಕ್ಕೆ ಎಸ್ ಇವರ ಮೊದಲು ಶ್ರೀ ಮಾಡುವ ಇದರಲ್ಲಿ ರೀತಿಯ ಮಾಡಿದ ಕಾಲ ಅಲ್ಲಿ ಮಾಡಲು ಅದೇ ಈಗ ಅವು ಗಳು ಎ ಎಂಬುದು ಅವನು ಅಂದರೆ ಅವರಿಗೆ ಇರುವ ವಿಶೇಷ ಮುಂದೆ ಅವುಗಳ ಮುಂತಾದ ಮೂಲ ಬಿ ಮೀ ಒಂದೇ ಇನ್ನೂ ಹೆಚ್ಚಾಗಿ ಮಾಡಿ ಅವರನ್ನು ಇದೇ ಯ ರೀತಿಯಲ್ಲಿ ಜೊತೆ ಅದರಲ್ಲಿ ಮಾಡಿದರು ನಡೆದ ಆಗ ಮತ್ತೆ ಪೂರ್ವ ಆತ ಬಂದ ಯಾವ ಒಟ್ಟು ಇತರೆ ಹಿಂದೆ ಪ್ರಮಾಣದ ಗಳನ್ನು ಕುರಿತು ಯು ಆದ್ದರಿಂದ ಅಲ್ಲದೆ ನಗರದ ಮೇಲಿನ ಏಕೆಂದರೆ ರಷ್ಟು ಎಂಬುದನ್ನು ಬಾರಿ ಎಂದರೆ ಹಿಂದಿನ ಆದರೂ ಆದ ಸಂಬಂಧಿಸಿದ ಮತ್ತೊಂದು ಸಿ ಆತನ ".split(" ")),e.kn.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.kn.tokenizer=function(t){if(!arguments.length||null==t||void 0==t)return[];if(Array.isArray(t))return t.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var n=t.toString().toLowerCase().replace(/^\s+/,"");return r.cut(n).split("|")},e.Pipeline.registerFunction(e.kn.stemmer,"stemmer-kn"),e.Pipeline.registerFunction(e.kn.stopWordFilter,"stopWordFilter-kn")}});
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){e.multiLanguage=function(){for(var t=Array.prototype.slice.call(arguments),i=t.join("-"),r="",n=[],s=[],p=0;p<t.length;++p)"en"==t[p]?(r+="\\w",n.unshift(e.stopWordFilter),n.push(e.stemmer),s.push(e.stemmer)):(r+=e[t[p]].wordCharacters,e[t[p]].stopWordFilter&&n.unshift(e[t[p]].stopWordFilter),e[t[p]].stemmer&&(n.push(e[t[p]].stemmer),s.push(e[t[p]].stemmer)));var o=e.trimmerSupport.generateTrimmer(r);return e.Pipeline.registerFunction(o,"lunr-multi-trimmer-"+i),n.unshift(o),function(){this.pipeline.reset(),this.pipeline.add.apply(this.pipeline,n),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add.apply(this.searchPipeline,s))}}}});
File diff suppressed because one or more lines are too long
-18
View File
@@ -1,18 +0,0 @@
/*!
* Lunr languages, `Norwegian` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.no=function(){this.pipeline.reset(),this.pipeline.add(e.no.trimmer,e.no.stopWordFilter,e.no.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.no.stemmer))},e.no.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.no.trimmer=e.trimmerSupport.generateTrimmer(e.no.wordCharacters),e.Pipeline.registerFunction(e.no.trimmer,"trimmer-no"),e.no.stemmer=function(){var r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,i=new function(){function e(){var e,r=w.cursor+3;if(a=w.limit,0<=r||r<=w.limit){for(s=r;;){if(e=w.cursor,w.in_grouping(d,97,248)){w.cursor=e;break}if(e>=w.limit)return;w.cursor=e+1}for(;!w.out_grouping(d,97,248);){if(w.cursor>=w.limit)return;w.cursor++}a=w.cursor,a<s&&(a=s)}}function i(){var e,r,n;if(w.cursor>=a&&(r=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,e=w.find_among_b(m,29),w.limit_backward=r,e))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:n=w.limit-w.cursor,w.in_grouping_b(c,98,122)?w.slice_del():(w.cursor=w.limit-n,w.eq_s_b(1,"k")&&w.out_grouping_b(d,97,248)&&w.slice_del());break;case 3:w.slice_from("er")}}function t(){var e,r=w.limit-w.cursor;w.cursor>=a&&(e=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,w.find_among_b(u,2)?(w.bra=w.cursor,w.limit_backward=e,w.cursor=w.limit-r,w.cursor>w.limit_backward&&(w.cursor--,w.bra=w.cursor,w.slice_del())):w.limit_backward=e)}function o(){var e,r;w.cursor>=a&&(r=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,e=w.find_among_b(l,11),e?(w.bra=w.cursor,w.limit_backward=r,1==e&&w.slice_del()):w.limit_backward=r)}var s,a,m=[new r("a",-1,1),new r("e",-1,1),new r("ede",1,1),new r("ande",1,1),new r("ende",1,1),new r("ane",1,1),new r("ene",1,1),new r("hetene",6,1),new r("erte",1,3),new r("en",-1,1),new r("heten",9,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",12,1),new r("s",-1,2),new r("as",14,1),new r("es",14,1),new r("edes",16,1),new r("endes",16,1),new r("enes",16,1),new r("hetenes",19,1),new r("ens",14,1),new r("hetens",21,1),new r("ers",14,1),new r("ets",14,1),new r("et",-1,1),new r("het",25,1),new r("ert",-1,3),new r("ast",-1,1)],u=[new r("dt",-1,-1),new r("vt",-1,-1)],l=[new r("leg",-1,1),new r("eleg",0,1),new r("ig",-1,1),new r("eig",2,1),new r("lig",2,1),new r("elig",4,1),new r("els",-1,1),new r("lov",-1,1),new r("elov",7,1),new r("slov",7,1),new r("hetslov",9,1)],d=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],c=[119,125,149,1],w=new n;this.setCurrent=function(e){w.setCurrent(e)},this.getCurrent=function(){return w.getCurrent()},this.stem=function(){var r=w.cursor;return e(),w.limit_backward=r,w.cursor=w.limit,i(),w.cursor=w.limit,t(),w.cursor=w.limit,o(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return i.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}}(),e.Pipeline.registerFunction(e.no.stemmer,"stemmer-no"),e.no.stopWordFilter=e.generateStopWordFilter("alle at av bare begge ble blei bli blir blitt både båe da de deg dei deim deira deires dem den denne der dere deres det dette di din disse ditt du dykk dykkar då eg ein eit eitt eller elles en enn er et ett etter for fordi fra før ha hadde han hans har hennar henne hennes her hjå ho hoe honom hoss hossen hun hva hvem hver hvilke hvilken hvis hvor hvordan hvorfor i ikke ikkje ikkje ingen ingi inkje inn inni ja jeg kan kom korleis korso kun kunne kva kvar kvarhelst kven kvi kvifor man mange me med medan meg meget mellom men mi min mine mitt mot mykje ned no noe noen noka noko nokon nokor nokre nå når og også om opp oss over på samme seg selv si si sia sidan siden sin sine sitt sjøl skal skulle slik so som som somme somt så sånn til um upp ut uten var vart varte ved vere verte vi vil ville vore vors vort vår være være vært å".split(" ")),e.Pipeline.registerFunction(e.no.stopWordFilter,"stopWordFilter-no")}});
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.sa=function(){this.pipeline.reset(),this.pipeline.add(e.sa.trimmer,e.sa.stopWordFilter,e.sa.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sa.stemmer))},e.sa.wordCharacters="ऀ-ःऄ-एऐ-टठ-यर-िी-ॏॐ-य़ॠ-९॰-ॿ꣠-꣱ꣲ-ꣷ꣸-ꣻ꣼-ꣽꣾ-ꣿᆰ0-ᆰ9",e.sa.trimmer=e.trimmerSupport.generateTrimmer(e.sa.wordCharacters),e.Pipeline.registerFunction(e.sa.trimmer,"trimmer-sa"),e.sa.stopWordFilter=e.generateStopWordFilter('तथा अयम्‌ एकम्‌ इत्यस्मिन्‌ तथा तत्‌ वा अयम्‌ इत्यस्य ते आहूत उपरि तेषाम्‌ किन्तु तेषाम्‌ तदा इत्यनेन अधिकः इत्यस्य तत्‌ केचन बहवः द्वि तथा महत्वपूर्णः अयम्‌ अस्य विषये अयं अस्ति तत्‌ प्रथमः विषये इत्युपरि इत्युपरि इतर अधिकतमः अधिकः अपि सामान्यतया ठ इतरेतर नूतनम्‌ द न्यूनम्‌ कश्चित्‌ वा विशालः द सः अस्ति तदनुसारम् तत्र अस्ति केवलम्‌ अपि अत्र सर्वे विविधाः तत्‌ बहवः यतः इदानीम्‌ द दक्षिण इत्यस्मै तस्य उपरि नथ अतीव कार्यम्‌ सर्वे एकैकम्‌ इत्यादि। एते सन्ति उत इत्थम्‌ मध्ये एतदर्थं . स कस्य प्रथमः श्री. करोति अस्मिन् प्रकारः निर्मिता कालः तत्र कर्तुं समान अधुना ते सन्ति स एकः अस्ति सः अर्थात् तेषां कृते . स्थितम् विशेषः अग्रिम तेषाम्‌ समान स्रोतः ख म समान इदानीमपि अधिकतया करोतु ते समान इत्यस्य वीथी सह यस्मिन् कृतवान्‌ धृतः तदा पुनः पूर्वं सः आगतः किम्‌ कुल इतर पुरा मात्रा स विषये उ अतएव अपि नगरस्य उपरि यतः प्रतिशतं कतरः कालः साधनानि भूत तथापि जात सम्बन्धि अन्यत्‌ ग अतः अस्माकं स्वकीयाः अस्माकं इदानीं अन्तः इत्यादयः भवन्तः इत्यादयः एते एताः तस्य अस्य इदम् एते तेषां तेषां तेषां तान् तेषां तेषां तेषां समानः सः एकः च तादृशाः बहवः अन्ये च वदन्ति यत् कियत् कस्मै कस्मै यस्मै यस्मै यस्मै यस्मै न अतिनीचः किन्तु प्रथमं सम्पूर्णतया ततः चिरकालानन्तरं पुस्तकं सम्पूर्णतया अन्तः किन्तु अत्र वा इह इव श्रद्धाय अवशिष्यते परन्तु अन्ये वर्गाः सन्ति ते सन्ति शक्नुवन्ति सर्वे मिलित्वा सर्वे एकत्र"'.split(" ")),e.sa.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.sa.tokenizer=function(t){if(!arguments.length||null==t||void 0==t)return[];if(Array.isArray(t))return t.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var i=t.toString().toLowerCase().replace(/^\s+/,"");return r.cut(i).split("|")},e.Pipeline.registerFunction(e.sa.stemmer,"stemmer-sa"),e.Pipeline.registerFunction(e.sa.stopWordFilter,"stopWordFilter-sa")}});
@@ -1 +0,0 @@
!function(r,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(r.lunr)}(this,function(){return function(r){r.stemmerSupport={Among:function(r,t,i,s){if(this.toCharArray=function(r){for(var t=r.length,i=new Array(t),s=0;s<t;s++)i[s]=r.charCodeAt(s);return i},!r&&""!=r||!t&&0!=t||!i)throw"Bad Among initialisation: s:"+r+", substring_i: "+t+", result: "+i;this.s_size=r.length,this.s=this.toCharArray(r),this.substring_i=t,this.result=i,this.method=s},SnowballProgram:function(){var r;return{bra:0,ket:0,limit:0,cursor:0,limit_backward:0,setCurrent:function(t){r=t,this.cursor=0,this.limit=t.length,this.limit_backward=0,this.bra=this.cursor,this.ket=this.limit},getCurrent:function(){var t=r;return r=null,t},in_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e<=s&&e>=i&&(e-=i,t[e>>3]&1<<(7&e)))return this.cursor++,!0}return!1},in_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e<=s&&e>=i&&(e-=i,t[e>>3]&1<<(7&e)))return this.cursor--,!0}return!1},out_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e>s||e<i)return this.cursor++,!0;if(e-=i,!(t[e>>3]&1<<(7&e)))return this.cursor++,!0}return!1},out_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e>s||e<i)return this.cursor--,!0;if(e-=i,!(t[e>>3]&1<<(7&e)))return this.cursor--,!0}return!1},eq_s:function(t,i){if(this.limit-this.cursor<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor+s)!=i.charCodeAt(s))return!1;return this.cursor+=t,!0},eq_s_b:function(t,i){if(this.cursor-this.limit_backward<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor-t+s)!=i.charCodeAt(s))return!1;return this.cursor-=t,!0},find_among:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=t[a],m=l;m<_.s_size;m++){if(n+l==u){f=-1;break}if(f=r.charCodeAt(n+l)-_.s[m])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){var _=t[s];if(o>=_.s_size){if(this.cursor=n+_.s_size,!_.method)return _.result;var b=_.method();if(this.cursor=n+_.s_size,b)return _.result}if((s=_.substring_i)<0)return 0}},find_among_b:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit_backward,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=t[a],m=_.s_size-1-l;m>=0;m--){if(n-l==u){f=-1;break}if(f=r.charCodeAt(n-1-l)-_.s[m])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){var _=t[s];if(o>=_.s_size){if(this.cursor=n-_.s_size,!_.method)return _.result;var b=_.method();if(this.cursor=n-_.s_size,b)return _.result}if((s=_.substring_i)<0)return 0}},replace_s:function(t,i,s){var e=s.length-(i-t),n=r.substring(0,t),u=r.substring(i);return r=n+s+u,this.limit+=e,this.cursor>=i?this.cursor+=e:this.cursor>t&&(this.cursor=t),e},slice_check:function(){if(this.bra<0||this.bra>this.ket||this.ket>this.limit||this.limit>r.length)throw"faulty slice operation"},slice_from:function(r){this.slice_check(),this.replace_s(this.bra,this.ket,r)},slice_del:function(){this.slice_from("")},insert:function(r,t,i){var s=this.replace_s(r,t,i);r<=this.bra&&(this.bra+=s),r<=this.ket&&(this.ket+=s)},slice_to:function(){return this.slice_check(),r.substring(this.bra,this.ket)},eq_v_b:function(r){return this.eq_s_b(r.length,r)}}}},r.trimmerSupport={generateTrimmer:function(r){var t=new RegExp("^[^"+r+"]+"),i=new RegExp("[^"+r+"]+$");return function(r){return"function"==typeof r.update?r.update(function(r){return r.replace(t,"").replace(i,"")}):r.replace(t,"").replace(i,"")}}}}});
-18
View File
@@ -1,18 +0,0 @@
/*!
* Lunr languages, `Swedish` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.sv=function(){this.pipeline.reset(),this.pipeline.add(e.sv.trimmer,e.sv.stopWordFilter,e.sv.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sv.stemmer))},e.sv.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.sv.trimmer=e.trimmerSupport.generateTrimmer(e.sv.wordCharacters),e.Pipeline.registerFunction(e.sv.trimmer,"trimmer-sv"),e.sv.stemmer=function(){var r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,t=new function(){function e(){var e,r=w.cursor+3;if(o=w.limit,0<=r||r<=w.limit){for(a=r;;){if(e=w.cursor,w.in_grouping(l,97,246)){w.cursor=e;break}if(w.cursor=e,w.cursor>=w.limit)return;w.cursor++}for(;!w.out_grouping(l,97,246);){if(w.cursor>=w.limit)return;w.cursor++}o=w.cursor,o<a&&(o=a)}}function t(){var e,r=w.limit_backward;if(w.cursor>=o&&(w.limit_backward=o,w.cursor=w.limit,w.ket=w.cursor,e=w.find_among_b(u,37),w.limit_backward=r,e))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:w.in_grouping_b(d,98,121)&&w.slice_del()}}function i(){var e=w.limit_backward;w.cursor>=o&&(w.limit_backward=o,w.cursor=w.limit,w.find_among_b(c,7)&&(w.cursor=w.limit,w.ket=w.cursor,w.cursor>w.limit_backward&&(w.bra=--w.cursor,w.slice_del())),w.limit_backward=e)}function s(){var e,r;if(w.cursor>=o){if(r=w.limit_backward,w.limit_backward=o,w.cursor=w.limit,w.ket=w.cursor,e=w.find_among_b(m,5))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:w.slice_from("lös");break;case 3:w.slice_from("full")}w.limit_backward=r}}var a,o,u=[new r("a",-1,1),new r("arna",0,1),new r("erna",0,1),new r("heterna",2,1),new r("orna",0,1),new r("ad",-1,1),new r("e",-1,1),new r("ade",6,1),new r("ande",6,1),new r("arne",6,1),new r("are",6,1),new r("aste",6,1),new r("en",-1,1),new r("anden",12,1),new r("aren",12,1),new r("heten",12,1),new r("ern",-1,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",18,1),new r("or",-1,1),new r("s",-1,2),new r("as",21,1),new r("arnas",22,1),new r("ernas",22,1),new r("ornas",22,1),new r("es",21,1),new r("ades",26,1),new r("andes",26,1),new r("ens",21,1),new r("arens",29,1),new r("hetens",29,1),new r("erns",21,1),new r("at",-1,1),new r("andet",-1,1),new r("het",-1,1),new r("ast",-1,1)],c=[new r("dd",-1,-1),new r("gd",-1,-1),new r("nn",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1),new r("tt",-1,-1)],m=[new r("ig",-1,1),new r("lig",0,1),new r("els",-1,1),new r("fullt",-1,3),new r("löst",-1,2)],l=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,24,0,32],d=[119,127,149],w=new n;this.setCurrent=function(e){w.setCurrent(e)},this.getCurrent=function(){return w.getCurrent()},this.stem=function(){var r=w.cursor;return e(),w.limit_backward=r,w.cursor=w.limit,t(),w.cursor=w.limit,i(),w.cursor=w.limit,s(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return t.setCurrent(e),t.stem(),t.getCurrent()}):(t.setCurrent(e),t.stem(),t.getCurrent())}}(),e.Pipeline.registerFunction(e.sv.stemmer,"stemmer-sv"),e.sv.stopWordFilter=e.generateStopWordFilter("alla allt att av blev bli blir blivit de dem den denna deras dess dessa det detta dig din dina ditt du där då efter ej eller en er era ert ett från för ha hade han hans har henne hennes hon honom hur här i icke ingen inom inte jag ju kan kunde man med mellan men mig min mina mitt mot mycket ni nu när någon något några och om oss på samma sedan sig sin sina sitta själv skulle som så sådan sådana sådant till under upp ut utan vad var vara varför varit varje vars vart vem vi vid vilka vilkas vilken vilket vår våra vårt än är åt över".split(" ")),e.Pipeline.registerFunction(e.sv.stopWordFilter,"stopWordFilter-sv")}});
-1
View File
@@ -1 +0,0 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.ta=function(){this.pipeline.reset(),this.pipeline.add(e.ta.trimmer,e.ta.stopWordFilter,e.ta.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.ta.stemmer))},e.ta.wordCharacters="஀-உஊ-ஏஐ-ஙச-ட஠-னப-யர-ஹ஺-ிீ-௉ொ-௏ௐ-௙௚-௟௠-௩௪-௯௰-௹௺-௿a-zA-Z-zA-0-9-",e.ta.trimmer=e.trimmerSupport.generateTrimmer(e.ta.wordCharacters),e.Pipeline.registerFunction(e.ta.trimmer,"trimmer-ta"),e.ta.stopWordFilter=e.generateStopWordFilter("அங்கு அங்கே அது அதை அந்த அவர் அவர்கள் அவள் அவன் அவை ஆக ஆகவே ஆகையால் ஆதலால் ஆதலினால் ஆனாலும் ஆனால் இங்கு இங்கே இது இதை இந்த இப்படி இவர் இவர்கள் இவள் இவன் இவை இவ்வளவு உனக்கு உனது உன் உன்னால் எங்கு எங்கே எது எதை எந்த எப்படி எவர் எவர்கள் எவள் எவன் எவை எவ்வளவு எனக்கு எனது எனவே என் என்ன என்னால் ஏது ஏன் தனது தன்னால் தானே தான் நாங்கள் நாம் நான் நீ நீங்கள்".split(" ")),e.ta.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var t=e.wordcut;t.init(),e.ta.tokenizer=function(r){if(!arguments.length||null==r||void 0==r)return[];if(Array.isArray(r))return r.map(function(t){return isLunr2?new e.Token(t.toLowerCase()):t.toLowerCase()});var i=r.toString().toLowerCase().replace(/^\s+/,"");return t.cut(i).split("|")},e.Pipeline.registerFunction(e.ta.stemmer,"stemmer-ta"),e.Pipeline.registerFunction(e.ta.stopWordFilter,"stopWordFilter-ta")}});
-1
View File
@@ -1 +0,0 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.te=function(){this.pipeline.reset(),this.pipeline.add(e.te.trimmer,e.te.stopWordFilter,e.te.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.te.stemmer))},e.te.wordCharacters="ఀ-ఄఅ-ఔక-హా-ౌౕ-ౖౘ-ౚౠ-ౡౢ-ౣ౦-౯౸-౿఼ఽ్ౝ౷౤౥",e.te.trimmer=e.trimmerSupport.generateTrimmer(e.te.wordCharacters),e.Pipeline.registerFunction(e.te.trimmer,"trimmer-te"),e.te.stopWordFilter=e.generateStopWordFilter("అందరూ అందుబాటులో అడగండి అడగడం అడ్డంగా అనుగుణంగా అనుమతించు అనుమతిస్తుంది అయితే ఇప్పటికే ఉన్నారు ఎక్కడైనా ఎప్పుడు ఎవరైనా ఎవరో ఏ ఏదైనా ఏమైనప్పటికి ఒక ఒకరు కనిపిస్తాయి కాదు కూడా గా గురించి చుట్టూ చేయగలిగింది తగిన తర్వాత దాదాపు దూరంగా నిజంగా పై ప్రకారం ప్రక్కన మధ్య మరియు మరొక మళ్ళీ మాత్రమే మెచ్చుకో వద్ద వెంట వేరుగా వ్యతిరేకంగా సంబంధం".split(" ")),e.te.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var t=e.wordcut;t.init(),e.te.tokenizer=function(r){if(!arguments.length||null==r||void 0==r)return[];if(Array.isArray(r))return r.map(function(t){return isLunr2?new e.Token(t.toLowerCase()):t.toLowerCase()});var i=r.toString().toLowerCase().replace(/^\s+/,"");return t.cut(i).split("|")},e.Pipeline.registerFunction(e.te.stemmer,"stemmer-te"),e.Pipeline.registerFunction(e.te.stopWordFilter,"stopWordFilter-te")}});
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.th=function(){this.pipeline.reset(),this.pipeline.add(e.th.trimmer),r?this.tokenizer=e.th.tokenizer:(e.tokenizer&&(e.tokenizer=e.th.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.th.tokenizer))},e.th.wordCharacters="[฀-๿]",e.th.trimmer=e.trimmerSupport.generateTrimmer(e.th.wordCharacters),e.Pipeline.registerFunction(e.th.trimmer,"trimmer-th");var t=e.wordcut;t.init(),e.th.tokenizer=function(i){if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(t){return r?new e.Token(t):t});var n=i.toString().replace(/^\s+/,"");return t.cut(n).split("|")}}});
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.vi=function(){this.pipeline.reset(),this.pipeline.add(e.vi.stopWordFilter,e.vi.trimmer)},e.vi.wordCharacters="[A-Za-ẓ̀͐́͑̉̃̓ÂâÊêÔôĂ-ăĐ-đƠ-ơƯ-ư]",e.vi.trimmer=e.trimmerSupport.generateTrimmer(e.vi.wordCharacters),e.Pipeline.registerFunction(e.vi.trimmer,"trimmer-vi"),e.vi.stopWordFilter=e.generateStopWordFilter("là cái nhưng mà".split(" "))}});
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r(require("@node-rs/jieba")):r()(e.lunr)}(this,function(e){return function(r,t){if(void 0===r)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===r.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var i="2"==r.version[0];r.zh=function(){this.pipeline.reset(),this.pipeline.add(r.zh.trimmer,r.zh.stopWordFilter,r.zh.stemmer),i?this.tokenizer=r.zh.tokenizer:(r.tokenizer&&(r.tokenizer=r.zh.tokenizer),this.tokenizerFn&&(this.tokenizerFn=r.zh.tokenizer))},r.zh.tokenizer=function(n){if(!arguments.length||null==n||void 0==n)return[];if(Array.isArray(n))return n.map(function(e){return i?new r.Token(e.toLowerCase()):e.toLowerCase()});t&&e.load(t);var o=n.toString().trim().toLowerCase(),s=[];e.cut(o,!0).forEach(function(e){s=s.concat(e.split(" "))}),s=s.filter(function(e){return!!e});var u=0;return s.map(function(e,t){if(i){var n=o.indexOf(e,u),s={};return s.position=[n,e.length],s.index=t,u=n,new r.Token(e,s)}return e})},r.zh.wordCharacters="\\w一-龥",r.zh.trimmer=r.trimmerSupport.generateTrimmer(r.zh.wordCharacters),r.Pipeline.registerFunction(r.zh.trimmer,"trimmer-zh"),r.zh.stemmer=function(){return function(e){return e}}(),r.Pipeline.registerFunction(r.zh.stemmer,"stemmer-zh"),r.zh.stopWordFilter=r.generateStopWordFilter("的 一 不 在 人 有 是 为 為 以 于 於 上 他 而 后 後 之 来 來 及 了 因 下 可 到 由 这 這 与 與 也 此 但 并 並 个 個 其 已 无 無 小 我 们 們 起 最 再 今 去 好 只 又 或 很 亦 某 把 那 你 乃 它 吧 被 比 别 趁 当 當 从 從 得 打 凡 儿 兒 尔 爾 该 該 各 给 給 跟 和 何 还 還 即 几 幾 既 看 据 據 距 靠 啦 另 么 麽 每 嘛 拿 哪 您 凭 憑 且 却 卻 让 讓 仍 啥 如 若 使 谁 誰 虽 雖 随 隨 同 所 她 哇 嗡 往 些 向 沿 哟 喲 用 咱 则 則 怎 曾 至 致 着 著 诸 諸 自".split(" ")),r.Pipeline.registerFunction(r.zh.stopWordFilter,"stopWordFilter-zh")}});
-206
View File
@@ -1,206 +0,0 @@
/**
* export the module via AMD, CommonJS or as a browser global
* Export code from https://github.com/umdjs/umd/blob/master/returnExports.js
*/
;(function (root, factory) {
if (typeof define === 'function' && define.amd) {
// AMD. Register as an anonymous module.
define(factory)
} else if (typeof exports === 'object') {
/**
* Node. Does not work with strict CommonJS, but
* only CommonJS-like environments that support module.exports,
* like Node.
*/
module.exports = factory()
} else {
// Browser globals (root is window)
factory()(root.lunr);
}
}(this, function () {
/**
* Just return a value to define the module export.
* This example returns an object, but the module
* can return a function as the exported value.
*/
return function(lunr) {
// TinySegmenter 0.1 -- Super compact Japanese tokenizer in Javascript
// (c) 2008 Taku Kudo <taku@chasen.org>
// TinySegmenter is freely distributable under the terms of a new BSD licence.
// For details, see http://chasen.org/~taku/software/TinySegmenter/LICENCE.txt
function TinySegmenter() {
var patterns = {
"[一二三四五六七八九十百千万億兆]":"M",
"[一-龠々〆ヵヶ]":"H",
"[ぁ-ん]":"I",
"[ァ-ヴーア-ン゙ー]":"K",
"[a-zA-Z-zA-]":"A",
"[0-9-]":"N"
}
this.chartype_ = [];
for (var i in patterns) {
var regexp = new RegExp(i);
this.chartype_.push([regexp, patterns[i]]);
}
this.BIAS__ = -332
this.BC1__ = {"HH":6,"II":2461,"KH":406,"OH":-1378};
this.BC2__ = {"AA":-3267,"AI":2744,"AN":-878,"HH":-4070,"HM":-1711,"HN":4012,"HO":3761,"IA":1327,"IH":-1184,"II":-1332,"IK":1721,"IO":5492,"KI":3831,"KK":-8741,"MH":-3132,"MK":3334,"OO":-2920};
this.BC3__ = {"HH":996,"HI":626,"HK":-721,"HN":-1307,"HO":-836,"IH":-301,"KK":2762,"MK":1079,"MM":4034,"OA":-1652,"OH":266};
this.BP1__ = {"BB":295,"OB":304,"OO":-125,"UB":352};
this.BP2__ = {"BO":60,"OO":-1762};
this.BQ1__ = {"BHH":1150,"BHM":1521,"BII":-1158,"BIM":886,"BMH":1208,"BNH":449,"BOH":-91,"BOO":-2597,"OHI":451,"OIH":-296,"OKA":1851,"OKH":-1020,"OKK":904,"OOO":2965};
this.BQ2__ = {"BHH":118,"BHI":-1159,"BHM":466,"BIH":-919,"BKK":-1720,"BKO":864,"OHH":-1139,"OHM":-181,"OIH":153,"UHI":-1146};
this.BQ3__ = {"BHH":-792,"BHI":2664,"BII":-299,"BKI":419,"BMH":937,"BMM":8335,"BNN":998,"BOH":775,"OHH":2174,"OHM":439,"OII":280,"OKH":1798,"OKI":-793,"OKO":-2242,"OMH":-2402,"OOO":11699};
this.BQ4__ = {"BHH":-3895,"BIH":3761,"BII":-4654,"BIK":1348,"BKK":-1806,"BMI":-3385,"BOO":-12396,"OAH":926,"OHH":266,"OHK":-2036,"ONN":-973};
this.BW1__ = {",と":660,",同":727,"B1あ":1404,"B1同":542,"、と":660,"、同":727,"」と":1682,"あっ":1505,"いう":1743,"いっ":-2055,"いる":672,"うし":-4817,"うん":665,"から":3472,"がら":600,"こう":-790,"こと":2083,"こん":-1262,"さら":-4143,"さん":4573,"した":2641,"して":1104,"すで":-3399,"そこ":1977,"それ":-871,"たち":1122,"ため":601,"った":3463,"つい":-802,"てい":805,"てき":1249,"でき":1127,"です":3445,"では":844,"とい":-4915,"とみ":1922,"どこ":3887,"ない":5713,"なっ":3015,"など":7379,"なん":-1113,"にし":2468,"には":1498,"にも":1671,"に対":-912,"の一":-501,"の中":741,"ませ":2448,"まで":1711,"まま":2600,"まる":-2155,"やむ":-1947,"よっ":-2565,"れた":2369,"れで":-913,"をし":1860,"を見":731,"亡く":-1886,"京都":2558,"取り":-2784,"大き":-2604,"大阪":1497,"平方":-2314,"引き":-1336,"日本":-195,"本当":-2423,"毎日":-2113,"目指":-724,"B1あ":1404,"B1同":542,"」と":1682};
this.BW2__ = {"..":-11822,"11":-669,"――":-5730,"−−":-13175,"いう":-1609,"うか":2490,"かし":-1350,"かも":-602,"から":-7194,"かれ":4612,"がい":853,"がら":-3198,"きた":1941,"くな":-1597,"こと":-8392,"この":-4193,"させ":4533,"され":13168,"さん":-3977,"しい":-1819,"しか":-545,"した":5078,"して":972,"しな":939,"その":-3744,"たい":-1253,"たた":-662,"ただ":-3857,"たち":-786,"たと":1224,"たは":-939,"った":4589,"って":1647,"っと":-2094,"てい":6144,"てき":3640,"てく":2551,"ては":-3110,"ても":-3065,"でい":2666,"でき":-1528,"でし":-3828,"です":-4761,"でも":-4203,"とい":1890,"とこ":-1746,"とと":-2279,"との":720,"とみ":5168,"とも":-3941,"ない":-2488,"なが":-1313,"など":-6509,"なの":2614,"なん":3099,"にお":-1615,"にし":2748,"にな":2454,"によ":-7236,"に対":-14943,"に従":-4688,"に関":-11388,"のか":2093,"ので":-7059,"のに":-6041,"のの":-6125,"はい":1073,"はが":-1033,"はず":-2532,"ばれ":1813,"まし":-1316,"まで":-6621,"まれ":5409,"めて":-3153,"もい":2230,"もの":-10713,"らか":-944,"らし":-1611,"らに":-1897,"りし":651,"りま":1620,"れた":4270,"れて":849,"れば":4114,"ろう":6067,"われ":7901,"を通":-11877,"んだ":728,"んな":-4115,"一人":602,"一方":-1375,"一日":970,"一部":-1051,"上が":-4479,"会社":-1116,"出て":2163,"分の":-7758,"同党":970,"同日":-913,"大阪":-2471,"委員":-1250,"少な":-1050,"年度":-8669,"年間":-1626,"府県":-2363,"手権":-1982,"新聞":-4066,"日新":-722,"日本":-7068,"日米":3372,"曜日":-601,"朝鮮":-2355,"本人":-2697,"東京":-1543,"然と":-1384,"社会":-1276,"立て":-990,"第に":-1612,"米国":-4268,"11":-669};
this.BW3__ = {"あた":-2194,"あり":719,"ある":3846,"い.":-1185,"い。":-1185,"いい":5308,"いえ":2079,"いく":3029,"いた":2056,"いっ":1883,"いる":5600,"いわ":1527,"うち":1117,"うと":4798,"えと":1454,"か.":2857,"か。":2857,"かけ":-743,"かっ":-4098,"かに":-669,"から":6520,"かり":-2670,"が,":1816,"が、":1816,"がき":-4855,"がけ":-1127,"がっ":-913,"がら":-4977,"がり":-2064,"きた":1645,"けど":1374,"こと":7397,"この":1542,"ころ":-2757,"さい":-714,"さを":976,"し,":1557,"し、":1557,"しい":-3714,"した":3562,"して":1449,"しな":2608,"しま":1200,"す.":-1310,"す。":-1310,"する":6521,"ず,":3426,"ず、":3426,"ずに":841,"そう":428,"た.":8875,"た。":8875,"たい":-594,"たの":812,"たり":-1183,"たる":-853,"だ.":4098,"だ。":4098,"だっ":1004,"った":-4748,"って":300,"てい":6240,"てお":855,"ても":302,"です":1437,"でに":-1482,"では":2295,"とう":-1387,"とし":2266,"との":541,"とも":-3543,"どう":4664,"ない":1796,"なく":-903,"など":2135,"に,":-1021,"に、":-1021,"にし":1771,"にな":1906,"には":2644,"の,":-724,"の、":-724,"の子":-1000,"は,":1337,"は、":1337,"べき":2181,"まし":1113,"ます":6943,"まっ":-1549,"まで":6154,"まれ":-793,"らし":1479,"られ":6820,"るる":3818,"れ,":854,"れ、":854,"れた":1850,"れて":1375,"れば":-3246,"れる":1091,"われ":-605,"んだ":606,"んで":798,"カ月":990,"会議":860,"入り":1232,"大会":2217,"始め":1681,"市":965,"新聞":-5055,"日,":974,"日、":974,"社会":2024,"カ月":990};
this.TC1__ = {"AAA":1093,"HHH":1029,"HHM":580,"HII":998,"HOH":-390,"HOM":-331,"IHI":1169,"IOH":-142,"IOI":-1015,"IOM":467,"MMH":187,"OOI":-1832};
this.TC2__ = {"HHO":2088,"HII":-1023,"HMM":-1154,"IHI":-1965,"KKH":703,"OII":-2649};
this.TC3__ = {"AAA":-294,"HHH":346,"HHI":-341,"HII":-1088,"HIK":731,"HOH":-1486,"IHH":128,"IHI":-3041,"IHO":-1935,"IIH":-825,"IIM":-1035,"IOI":-542,"KHH":-1216,"KKA":491,"KKH":-1217,"KOK":-1009,"MHH":-2694,"MHM":-457,"MHO":123,"MMH":-471,"NNH":-1689,"NNO":662,"OHO":-3393};
this.TC4__ = {"HHH":-203,"HHI":1344,"HHK":365,"HHM":-122,"HHN":182,"HHO":669,"HIH":804,"HII":679,"HOH":446,"IHH":695,"IHO":-2324,"IIH":321,"III":1497,"IIO":656,"IOO":54,"KAK":4845,"KKA":3386,"KKK":3065,"MHH":-405,"MHI":201,"MMH":-241,"MMM":661,"MOM":841};
this.TQ1__ = {"BHHH":-227,"BHHI":316,"BHIH":-132,"BIHH":60,"BIII":1595,"BNHH":-744,"BOHH":225,"BOOO":-908,"OAKK":482,"OHHH":281,"OHIH":249,"OIHI":200,"OIIH":-68};
this.TQ2__ = {"BIHH":-1401,"BIII":-1033,"BKAK":-543,"BOOO":-5591};
this.TQ3__ = {"BHHH":478,"BHHM":-1073,"BHIH":222,"BHII":-504,"BIIH":-116,"BIII":-105,"BMHI":-863,"BMHM":-464,"BOMH":620,"OHHH":346,"OHHI":1729,"OHII":997,"OHMH":481,"OIHH":623,"OIIH":1344,"OKAK":2792,"OKHH":587,"OKKA":679,"OOHH":110,"OOII":-685};
this.TQ4__ = {"BHHH":-721,"BHHM":-3604,"BHII":-966,"BIIH":-607,"BIII":-2181,"OAAA":-2763,"OAKK":180,"OHHH":-294,"OHHI":2446,"OHHO":480,"OHIH":-1573,"OIHH":1935,"OIHI":-493,"OIIH":626,"OIII":-4007,"OKAK":-8156};
this.TW1__ = {"につい":-4681,"東京都":2026};
this.TW2__ = {"ある程":-2049,"いった":-1256,"ころが":-2434,"しょう":3873,"その後":-4430,"だって":-1049,"ていた":1833,"として":-4657,"ともに":-4517,"もので":1882,"一気に":-792,"初めて":-1512,"同時に":-8097,"大きな":-1255,"対して":-2721,"社会党":-3216};
this.TW3__ = {"いただ":-1734,"してい":1314,"として":-4314,"につい":-5483,"にとっ":-5989,"に当た":-6247,"ので,":-727,"ので、":-727,"のもの":-600,"れから":-3752,"十二月":-2287};
this.TW4__ = {"いう.":8576,"いう。":8576,"からな":-2348,"してい":2958,"たが,":1516,"たが、":1516,"ている":1538,"という":1349,"ました":5543,"ません":1097,"ようと":-4258,"よると":5865};
this.UC1__ = {"A":484,"K":93,"M":645,"O":-505};
this.UC2__ = {"A":819,"H":1059,"I":409,"M":3987,"N":5775,"O":646};
this.UC3__ = {"A":-1370,"I":2311};
this.UC4__ = {"A":-2643,"H":1809,"I":-1032,"K":-3450,"M":3565,"N":3876,"O":6646};
this.UC5__ = {"H":313,"I":-1238,"K":-799,"M":539,"O":-831};
this.UC6__ = {"H":-506,"I":-253,"K":87,"M":247,"O":-387};
this.UP1__ = {"O":-214};
this.UP2__ = {"B":69,"O":935};
this.UP3__ = {"B":189};
this.UQ1__ = {"BH":21,"BI":-12,"BK":-99,"BN":142,"BO":-56,"OH":-95,"OI":477,"OK":410,"OO":-2422};
this.UQ2__ = {"BH":216,"BI":113,"OK":1759};
this.UQ3__ = {"BA":-479,"BH":42,"BI":1913,"BK":-7198,"BM":3160,"BN":6427,"BO":14761,"OI":-827,"ON":-3212};
this.UW1__ = {",":156,"、":156,"「":-463,"あ":-941,"う":-127,"が":-553,"き":121,"こ":505,"で":-201,"と":-547,"ど":-123,"に":-789,"の":-185,"は":-847,"も":-466,"や":-470,"よ":182,"ら":-292,"り":208,"れ":169,"を":-446,"ん":-137,"・":-135,"主":-402,"京":-268,"区":-912,"午":871,"国":-460,"大":561,"委":729,"市":-411,"日":-141,"理":361,"生":-408,"県":-386,"都":-718,"「":-463,"・":-135};
this.UW2__ = {",":-829,"、":-829,"":892,"「":-645,"」":3145,"あ":-538,"い":505,"う":134,"お":-502,"か":1454,"が":-856,"く":-412,"こ":1141,"さ":878,"ざ":540,"し":1529,"す":-675,"せ":300,"そ":-1011,"た":188,"だ":1837,"つ":-949,"て":-291,"で":-268,"と":-981,"ど":1273,"な":1063,"に":-1764,"の":130,"は":-409,"ひ":-1273,"べ":1261,"ま":600,"も":-1263,"や":-402,"よ":1639,"り":-579,"る":-694,"れ":571,"を":-2516,"ん":2095,"ア":-587,"カ":306,"キ":568,"ッ":831,"三":-758,"不":-2150,"世":-302,"中":-968,"主":-861,"事":492,"人":-123,"会":978,"保":362,"入":548,"初":-3025,"副":-1566,"北":-3414,"区":-422,"大":-1769,"天":-865,"太":-483,"子":-1519,"学":760,"実":1023,"小":-2009,"市":-813,"年":-1060,"強":1067,"手":-1519,"揺":-1033,"政":1522,"文":-1355,"新":-1682,"日":-1815,"明":-1462,"最":-630,"朝":-1843,"本":-1650,"東":-931,"果":-665,"次":-2378,"民":-180,"気":-1740,"理":752,"発":529,"目":-1584,"相":-242,"県":-1165,"立":-763,"第":810,"米":509,"自":-1353,"行":838,"西":-744,"見":-3874,"調":1010,"議":1198,"込":3041,"開":1758,"間":-1257,"「":-645,"」":3145,"ッ":831,"ア":-587,"カ":306,"キ":568};
this.UW3__ = {",":4889,"1":-800,"":-1723,"、":4889,"々":-2311,"":5827,"」":2670,"〓":-3573,"あ":-2696,"い":1006,"う":2342,"え":1983,"お":-4864,"か":-1163,"が":3271,"く":1004,"け":388,"げ":401,"こ":-3552,"ご":-3116,"さ":-1058,"し":-395,"す":584,"せ":3685,"そ":-5228,"た":842,"ち":-521,"っ":-1444,"つ":-1081,"て":6167,"で":2318,"と":1691,"ど":-899,"な":-2788,"に":2745,"の":4056,"は":4555,"ひ":-2171,"ふ":-1798,"へ":1199,"ほ":-5516,"ま":-4384,"み":-120,"め":1205,"も":2323,"や":-788,"よ":-202,"ら":727,"り":649,"る":5905,"れ":2773,"わ":-1207,"を":6620,"ん":-518,"ア":551,"グ":1319,"ス":874,"ッ":-1350,"ト":521,"ム":1109,"ル":1591,"ロ":2201,"ン":278,"・":-3794,"一":-1619,"下":-1759,"世":-2087,"両":3815,"中":653,"主":-758,"予":-1193,"二":974,"人":2742,"今":792,"他":1889,"以":-1368,"低":811,"何":4265,"作":-361,"保":-2439,"元":4858,"党":3593,"全":1574,"公":-3030,"六":755,"共":-1880,"円":5807,"再":3095,"分":457,"初":2475,"別":1129,"前":2286,"副":4437,"力":365,"動":-949,"務":-1872,"化":1327,"北":-1038,"区":4646,"千":-2309,"午":-783,"協":-1006,"口":483,"右":1233,"各":3588,"合":-241,"同":3906,"和":-837,"員":4513,"国":642,"型":1389,"場":1219,"外":-241,"妻":2016,"学":-1356,"安":-423,"実":-1008,"家":1078,"小":-513,"少":-3102,"州":1155,"市":3197,"平":-1804,"年":2416,"広":-1030,"府":1605,"度":1452,"建":-2352,"当":-3885,"得":1905,"思":-1291,"性":1822,"戸":-488,"指":-3973,"政":-2013,"教":-1479,"数":3222,"文":-1489,"新":1764,"日":2099,"旧":5792,"昨":-661,"時":-1248,"曜":-951,"最":-937,"月":4125,"期":360,"李":3094,"村":364,"東":-805,"核":5156,"森":2438,"業":484,"氏":2613,"民":-1694,"決":-1073,"法":1868,"海":-495,"無":979,"物":461,"特":-3850,"生":-273,"用":914,"町":1215,"的":7313,"直":-1835,"省":792,"県":6293,"知":-1528,"私":4231,"税":401,"立":-960,"第":1201,"米":7767,"系":3066,"約":3663,"級":1384,"統":-4229,"総":1163,"線":1255,"者":6457,"能":725,"自":-2869,"英":785,"見":1044,"調":-562,"財":-733,"費":1777,"車":1835,"軍":1375,"込":-1504,"通":-1136,"選":-681,"郎":1026,"郡":4404,"部":1200,"金":2163,"長":421,"開":-1432,"間":1302,"関":-1282,"雨":2009,"電":-1045,"非":2066,"駅":1620,"":-800,"」":2670,"・":-3794,"ッ":-1350,"ア":551,"グ":1319,"ス":874,"ト":521,"ム":1109,"ル":1591,"ロ":2201,"ン":278};
this.UW4__ = {",":3930,".":3508,"―":-4841,"、":3930,"。":3508,"":4999,"「":1895,"」":3798,"〓":-5156,"あ":4752,"い":-3435,"う":-640,"え":-2514,"お":2405,"か":530,"が":6006,"き":-4482,"ぎ":-3821,"く":-3788,"け":-4376,"げ":-4734,"こ":2255,"ご":1979,"さ":2864,"し":-843,"じ":-2506,"す":-731,"ず":1251,"せ":181,"そ":4091,"た":5034,"だ":5408,"ち":-3654,"っ":-5882,"つ":-1659,"て":3994,"で":7410,"と":4547,"な":5433,"に":6499,"ぬ":1853,"ね":1413,"の":7396,"は":8578,"ば":1940,"ひ":4249,"び":-4134,"ふ":1345,"へ":6665,"べ":-744,"ほ":1464,"ま":1051,"み":-2082,"む":-882,"め":-5046,"も":4169,"ゃ":-2666,"や":2795,"ょ":-1544,"よ":3351,"ら":-2922,"り":-9726,"る":-14896,"れ":-2613,"ろ":-4570,"わ":-1783,"を":13150,"ん":-2352,"カ":2145,"コ":1789,"セ":1287,"ッ":-724,"ト":-403,"メ":-1635,"ラ":-881,"リ":-541,"ル":-856,"ン":-3637,"・":-4371,"ー":-11870,"一":-2069,"中":2210,"予":782,"事":-190,"井":-1768,"人":1036,"以":544,"会":950,"体":-1286,"作":530,"側":4292,"先":601,"党":-2006,"共":-1212,"内":584,"円":788,"初":1347,"前":1623,"副":3879,"力":-302,"動":-740,"務":-2715,"化":776,"区":4517,"協":1013,"参":1555,"合":-1834,"和":-681,"員":-910,"器":-851,"回":1500,"国":-619,"園":-1200,"地":866,"場":-1410,"塁":-2094,"士":-1413,"多":1067,"大":571,"子":-4802,"学":-1397,"定":-1057,"寺":-809,"小":1910,"屋":-1328,"山":-1500,"島":-2056,"川":-2667,"市":2771,"年":374,"庁":-4556,"後":456,"性":553,"感":916,"所":-1566,"支":856,"改":787,"政":2182,"教":704,"文":522,"方":-856,"日":1798,"時":1829,"最":845,"月":-9066,"木":-485,"来":-442,"校":-360,"業":-1043,"氏":5388,"民":-2716,"気":-910,"沢":-939,"済":-543,"物":-735,"率":672,"球":-1267,"生":-1286,"産":-1101,"田":-2900,"町":1826,"的":2586,"目":922,"省":-3485,"県":2997,"空":-867,"立":-2112,"第":788,"米":2937,"系":786,"約":2171,"経":1146,"統":-1169,"総":940,"線":-994,"署":749,"者":2145,"能":-730,"般":-852,"行":-792,"規":792,"警":-1184,"議":-244,"谷":-1000,"賞":730,"車":-1481,"軍":1158,"輪":-1433,"込":-3370,"近":929,"道":-1291,"選":2596,"郎":-4866,"都":1192,"野":-1100,"銀":-2213,"長":357,"間":-2344,"院":-2297,"際":-2604,"電":-878,"領":-1659,"題":-792,"館":-1984,"首":1749,"高":2120,"「":1895,"」":3798,"・":-4371,"ッ":-724,"ー":-11870,"カ":2145,"コ":1789,"セ":1287,"ト":-403,"メ":-1635,"ラ":-881,"リ":-541,"ル":-856,"ン":-3637};
this.UW5__ = {",":465,".":-299,"1":-514,"E2":-32768,"]":-2762,"、":465,"。":-299,"「":363,"あ":1655,"い":331,"う":-503,"え":1199,"お":527,"か":647,"が":-421,"き":1624,"ぎ":1971,"く":312,"げ":-983,"さ":-1537,"し":-1371,"す":-852,"だ":-1186,"ち":1093,"っ":52,"つ":921,"て":-18,"で":-850,"と":-127,"ど":1682,"な":-787,"に":-1224,"の":-635,"は":-578,"べ":1001,"み":502,"め":865,"ゃ":3350,"ょ":854,"り":-208,"る":429,"れ":504,"わ":419,"を":-1264,"ん":327,"イ":241,"ル":451,"ン":-343,"中":-871,"京":722,"会":-1153,"党":-654,"務":3519,"区":-901,"告":848,"員":2104,"大":-1296,"学":-548,"定":1785,"嵐":-1304,"市":-2991,"席":921,"年":1763,"思":872,"所":-814,"挙":1618,"新":-1682,"日":218,"月":-4353,"査":932,"格":1356,"機":-1508,"氏":-1347,"田":240,"町":-3912,"的":-3149,"相":1319,"省":-1052,"県":-4003,"研":-997,"社":-278,"空":-813,"統":1955,"者":-2233,"表":663,"語":-1073,"議":1219,"選":-1018,"郎":-368,"長":786,"間":1191,"題":2368,"館":-689,"":-514,"E2":-32768,"「":363,"イ":241,"ル":451,"ン":-343};
this.UW6__ = {",":227,".":808,"1":-270,"E1":306,"、":227,"。":808,"あ":-307,"う":189,"か":241,"が":-73,"く":-121,"こ":-200,"じ":1782,"す":383,"た":-428,"っ":573,"て":-1014,"で":101,"と":-105,"な":-253,"に":-149,"の":-417,"は":-236,"も":-206,"り":187,"る":-135,"を":195,"ル":-673,"ン":-496,"一":-277,"中":201,"件":-800,"会":624,"前":302,"区":1792,"員":-1212,"委":798,"学":-960,"市":887,"広":-695,"後":535,"業":-697,"相":753,"社":-507,"福":974,"空":-822,"者":1811,"連":463,"郎":1082,"":-270,"E1":306,"ル":-673,"ン":-496};
return this;
}
TinySegmenter.prototype.ctype_ = function(str) {
for (var i in this.chartype_) {
if (str.match(this.chartype_[i][0])) {
return this.chartype_[i][1];
}
}
return "O";
}
TinySegmenter.prototype.ts_ = function(v) {
if (v) { return v; }
return 0;
}
TinySegmenter.prototype.segment = function(input) {
if (input == null || input == undefined || input == "") {
return [];
}
var result = [];
var seg = ["B3","B2","B1"];
var ctype = ["O","O","O"];
var o = input.split("");
for (i = 0; i < o.length; ++i) {
seg.push(o[i]);
ctype.push(this.ctype_(o[i]))
}
seg.push("E1");
seg.push("E2");
seg.push("E3");
ctype.push("O");
ctype.push("O");
ctype.push("O");
var word = seg[3];
var p1 = "U";
var p2 = "U";
var p3 = "U";
for (var i = 4; i < seg.length - 3; ++i) {
var score = this.BIAS__;
var w1 = seg[i-3];
var w2 = seg[i-2];
var w3 = seg[i-1];
var w4 = seg[i];
var w5 = seg[i+1];
var w6 = seg[i+2];
var c1 = ctype[i-3];
var c2 = ctype[i-2];
var c3 = ctype[i-1];
var c4 = ctype[i];
var c5 = ctype[i+1];
var c6 = ctype[i+2];
score += this.ts_(this.UP1__[p1]);
score += this.ts_(this.UP2__[p2]);
score += this.ts_(this.UP3__[p3]);
score += this.ts_(this.BP1__[p1 + p2]);
score += this.ts_(this.BP2__[p2 + p3]);
score += this.ts_(this.UW1__[w1]);
score += this.ts_(this.UW2__[w2]);
score += this.ts_(this.UW3__[w3]);
score += this.ts_(this.UW4__[w4]);
score += this.ts_(this.UW5__[w5]);
score += this.ts_(this.UW6__[w6]);
score += this.ts_(this.BW1__[w2 + w3]);
score += this.ts_(this.BW2__[w3 + w4]);
score += this.ts_(this.BW3__[w4 + w5]);
score += this.ts_(this.TW1__[w1 + w2 + w3]);
score += this.ts_(this.TW2__[w2 + w3 + w4]);
score += this.ts_(this.TW3__[w3 + w4 + w5]);
score += this.ts_(this.TW4__[w4 + w5 + w6]);
score += this.ts_(this.UC1__[c1]);
score += this.ts_(this.UC2__[c2]);
score += this.ts_(this.UC3__[c3]);
score += this.ts_(this.UC4__[c4]);
score += this.ts_(this.UC5__[c5]);
score += this.ts_(this.UC6__[c6]);
score += this.ts_(this.BC1__[c2 + c3]);
score += this.ts_(this.BC2__[c3 + c4]);
score += this.ts_(this.BC3__[c4 + c5]);
score += this.ts_(this.TC1__[c1 + c2 + c3]);
score += this.ts_(this.TC2__[c2 + c3 + c4]);
score += this.ts_(this.TC3__[c3 + c4 + c5]);
score += this.ts_(this.TC4__[c4 + c5 + c6]);
// score += this.ts_(this.TC5__[c4 + c5 + c6]);
score += this.ts_(this.UQ1__[p1 + c1]);
score += this.ts_(this.UQ2__[p2 + c2]);
score += this.ts_(this.UQ3__[p3 + c3]);
score += this.ts_(this.BQ1__[p2 + c2 + c3]);
score += this.ts_(this.BQ2__[p2 + c3 + c4]);
score += this.ts_(this.BQ3__[p3 + c2 + c3]);
score += this.ts_(this.BQ4__[p3 + c3 + c4]);
score += this.ts_(this.TQ1__[p2 + c1 + c2 + c3]);
score += this.ts_(this.TQ2__[p2 + c2 + c3 + c4]);
score += this.ts_(this.TQ3__[p3 + c1 + c2 + c3]);
score += this.ts_(this.TQ4__[p3 + c2 + c3 + c4]);
var p = "O";
if (score > 0) {
result.push(word);
word = "";
p = "B";
}
p1 = p2;
p2 = p3;
p3 = p;
word += seg[i];
}
result.push(word);
return result;
}
lunr.TinySegmenter = TinySegmenter;
};
}));
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -1 +0,0 @@
{"version":3,"sources":["src/templates/assets/stylesheets/palette/_scheme.scss","../../../../src/templates/assets/stylesheets/palette.scss","src/templates/assets/stylesheets/palette/_accent.scss","src/templates/assets/stylesheets/palette/_primary.scss","src/templates/assets/stylesheets/utilities/_break.scss"],"names":[],"mappings":"AA2BA,cAGE,6BAME,sDAAA,CACA,6DAAA,CACA,+DAAA,CACA,gEAAA,CACA,mDAAA,CACA,6DAAA,CACA,+DAAA,CACA,gEAAA,CAGA,mDAAA,CACA,gDAAA,CACA,yDAAA,CACA,4DAAA,CAGA,0BAAA,CACA,mCAAA,CAGA,iCAAA,CACA,kCAAA,CACA,mCAAA,CACA,mCAAA,CACA,kCAAA,CACA,iCAAA,CACA,+CAAA,CACA,6DAAA,CACA,gEAAA,CACA,4DAAA,CACA,4DAAA,CACA,6DAAA,CAGA,6CAAA,CAGA,+CAAA,CAGA,uDAAA,CACA,6DAAA,CACA,2DAAA,CAGA,iCAAA,CAGA,yDAAA,CACA,iEAAA,CAGA,mDAAA,CACA,mDAAA,CAGA,qDAAA,CACA,uDAAA,CAGA,8DAAA,CAKA,8DAAA,CAKA,0DAAA,CAzEA,iBCiBF,CD6DE,kHAEE,YC3DJ,CDkFE,yDACE,4BChFJ,CD+EE,2DACE,4BC7EJ,CD4EE,gEACE,4BC1EJ,CDyEE,2DACE,4BCvEJ,CDsEE,yDACE,4BCpEJ,CDmEE,0DACE,4BCjEJ,CDgEE,gEACE,4BC9DJ,CD6DE,0DACE,4BC3DJ,CD0DE,2OACE,4BC/CJ,CDsDA,+FAGE,iCCpDF,CACF,CCjDE,2BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD6CN,CCvDE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDoDN,CC9DE,8BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD2DN,CCrEE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDkEN,CC5EE,8BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDyEN,CCnFE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDgFN,CC1FE,kCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDuFN,CCjGE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD8FN,CCxGE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDqGN,CC/GE,6BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD4GN,CCtHE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDmHN,CC7HE,4BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCD6HN,CCpIE,8BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCDoIN,CC3IE,6BACE,yBAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCD2IN,CClJE,8BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCDkJN,CCzJE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDsJN,CE3JE,4BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwJN,CEnKE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgKN,CE3KE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwKN,CEnLE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgLN,CE3LE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwLN,CEnME,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgMN,CE3ME,mCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwMN,CEnNE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgNN,CE3NE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwNN,CEnOE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgON,CE3OE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwON,CEnPE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCFmPN,CE3PE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCF2PN,CEnQE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCFmQN,CE3QE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCF2QN,CEnRE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgRN,CE3RE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwRN,CEnSE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCAAA,CAKA,4BF4RN,CE5SE,kCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCAAA,CAKA,4BFqSN,CEtRE,sEACE,4BFyRJ,CE1RE,+DACE,4BF6RJ,CE9RE,iEACE,4BFiSJ,CElSE,gEACE,4BFqSJ,CEtSE,iEACE,4BFySJ,CEhSA,8BACE,mDAAA,CACA,4DAAA,CACA,0DAAA,CACA,oDAAA,CACA,2DAAA,CAGA,4BFiSF,CE9RE,yCACE,+BFgSJ,CE7RI,kDAEE,0CAAA,CACA,sCAAA,CAFA,mCFiSN,CG7MI,mCD1EA,+CACE,8CF0RJ,CEvRI,qDACE,8CFyRN,CEpRE,iEACE,mCFsRJ,CACF,CGxNI,sCDvDA,uCACE,oCFkRJ,CACF,CEzQA,8BACE,kDAAA,CACA,4DAAA,CACA,wDAAA,CACA,oDAAA,CACA,6DAAA,CAGA,4BF0QF,CEvQE,yCACE,+BFyQJ,CEtQI,kDAEE,0CAAA,CACA,sCAAA,CAFA,mCF0QN,CEnQE,yCACE,6CFqQJ,CG9NI,0CDhCA,8CACE,gDFiQJ,CACF,CGnOI,0CDvBA,iFACE,6CF6PJ,CACF,CG3PI,sCDKA,uCACE,6CFyPJ,CACF","file":"palette.css"}
+14
View File
@@ -0,0 +1,14 @@
cmake_minimum_required(VERSION 3.21)
add_executable(bench_pipeline bench_pipeline.cpp)
target_link_libraries(bench_pipeline PRIVATE kpn)
target_compile_options(bench_pipeline PRIVATE -O3 -march=native)
find_package(TBB QUIET)
if(TBB_FOUND)
target_link_libraries(bench_pipeline PRIVATE TBB::tbb)
target_compile_definitions(bench_pipeline PRIVATE KPN_BENCH_TBB=1)
message(STATUS "TBB found — enabling TBB benchmarks")
else()
message(STATUS "TBB not found — TBB benchmarks disabled")
endif()
+526
View File
@@ -0,0 +1,526 @@
// Throughput benchmark: items/second vs. graph topology and size.
//
// Topologies:
// chain — linear depth D: push → n[0..D-1] → pop
// wide — fanout<W>: push → fanout → W parallel nodes → W pops
// diamond — push → fanout<2> → 2×2 nodes → 2 pops
//
// Two scheduling modes for each topology:
// private — each node owns a private ThreadPool(1) [Node<>]
// pool — all nodes share one ThreadPool(T) [PoolNode<> + shared pool]
//
// Usage: ./bench_pipeline | tee results.csv
#include <kpn/kpn.hpp>
#ifdef KPN_BENCH_TBB
#include <oneapi/tbb/flow_graph.h>
namespace tbb_flow = oneapi::tbb::flow;
#endif
#include <array>
#include <atomic>
#include <chrono>
#include <cstdio>
#include <memory>
#include <string>
#include <thread>
#include <vector>
using namespace kpn;
using namespace std::chrono_literals;
using sclock = std::chrono::steady_clock;
// ── configurable work ─────────────────────────────────────────────────────────
static std::atomic<int> g_work_us{0};
static int chain_fn(int x) {
int us = g_work_us.load(std::memory_order_relaxed);
if (us > 0) {
auto end = sclock::now() + std::chrono::microseconds(us);
while (sclock::now() < end);
}
return x;
}
using ChainNode = Node<chain_fn, in<>, out<>>;
using PoolChainNode = PoolNode<chain_fn, in<>, out<>>;
// ── push helper: yield-spin on overflow (no artificial sleep latency) ─────────
static void push_retry(Channel<int>& ch, int val) {
while (true) {
try { ch.push(val); return; }
catch (const ChannelOverflowError&) { std::this_thread::yield(); }
catch (const ChannelClosedError&) { return; }
}
}
// ── result ────────────────────────────────────────────────────────────────────
struct Result {
const char* topology;
int size;
int work_us;
int threads; // 0 = private (1 thread per node), N = shared pool size
double items_per_sec;
double overhead_us;
};
// ── chain ─────────────────────────────────────────────────────────────────────
static int items_for(int work_us, int depth = 1) {
int effective = std::max(1, work_us) * std::max(1, depth);
if (effective <= 1) return 5000;
if (effective <= 10) return 3000;
if (effective <= 100) return 1000;
if (effective <= 1000) return 200;
return 50;
}
static Result bench_chain(int depth, int work_us) {
const int N = items_for(work_us, depth);
const int CAP = N;
std::vector<std::shared_ptr<Channel<int>>> chs;
for (int i = 0; i <= depth; ++i)
chs.push_back(std::make_shared<Channel<int>>(CAP));
std::vector<std::unique_ptr<ChainNode>> nodes;
for (int i = 0; i < depth; ++i) {
nodes.push_back(std::make_unique<ChainNode>(CAP));
nodes.back()->set_input_channel<0>(chs[i]);
nodes.back()->set_output_channel<0>(chs[i + 1].get());
}
for (auto& n : nodes) n->start();
std::atomic<sclock::time_point> t1;
std::thread reader([&] {
for (int i = 0; i < N; ++i) chs.back()->pop();
t1.store(sclock::now(), std::memory_order_release);
});
auto t0 = sclock::now();
std::thread pusher([&] {
for (int i = 0; i < N; ++i) push_retry(*chs[0], i);
});
pusher.join();
reader.join();
for (auto& n : nodes) n->stop();
double elapsed = std::chrono::duration<double>(
t1.load(std::memory_order_acquire) - t0).count();
// Subtract theoretical pipeline fill cost (depth-1)*W so that overhead
// reflects only framework latency, not the expected pipeline startup time.
double pipeline_us = static_cast<double>(work_us) * (N + depth - 1);
double wus = (elapsed * 1e6 - pipeline_us) / N;
return {"chain", depth, work_us, 0, N / elapsed, wus};
}
static Result bench_chain_pool(int depth, int work_us, int pool_threads) {
const int N = items_for(work_us, depth);
const int CAP = N;
auto pool = std::make_shared<ThreadPool>(pool_threads);
std::vector<std::shared_ptr<Channel<int>>> chs;
for (int i = 0; i <= depth; ++i)
chs.push_back(std::make_shared<Channel<int>>(CAP));
std::vector<std::unique_ptr<PoolChainNode>> nodes;
for (int i = 0; i < depth; ++i) {
nodes.push_back(std::make_unique<PoolChainNode>(pool, CAP));
nodes.back()->set_input_channel<0>(chs[i]);
nodes.back()->set_output_channel<0>(chs[i + 1].get());
}
pool->start();
for (auto& n : nodes) n->start();
std::atomic<sclock::time_point> t1;
std::thread reader([&] {
for (int i = 0; i < N; ++i) chs.back()->pop();
t1.store(sclock::now(), std::memory_order_release);
});
auto t0 = sclock::now();
std::thread pusher([&] {
for (int i = 0; i < N; ++i) push_retry(*chs[0], i);
});
pusher.join();
reader.join();
for (auto& n : nodes) n->stop();
pool->stop();
double elapsed = std::chrono::duration<double>(
t1.load(std::memory_order_acquire) - t0).count();
double pipeline_us = static_cast<double>(work_us) * (N + depth - 1);
double wus = (elapsed * 1e6 - pipeline_us) / N;
return {"chain", depth, work_us, pool_threads, N / elapsed, wus};
}
// ── wide (fanout<W>) ──────────────────────────────────────────────────────────
template<std::size_t W>
static Result bench_wide(int work_us) {
const int N = items_for(work_us);
const int CAP = N;
auto src_ch = std::make_shared<Channel<int>>(CAP);
auto fan = std::make_unique<FanoutNode<int, W>>(CAP);
fan->template set_input_channel<0>(src_ch);
std::array<std::unique_ptr<ChainNode>, W> nodes;
std::array<std::shared_ptr<Channel<int>>, W> sink_chs;
for (std::size_t i = 0; i < W; ++i) {
nodes[i] = std::make_unique<ChainNode>(CAP);
sink_chs[i] = std::make_shared<Channel<int>>(CAP);
nodes[i]->template set_output_channel<0>(sink_chs[i].get());
}
[&]<std::size_t... Is>(std::index_sequence<Is...>) {
(fan->template set_output_channel<Is>(
&nodes[Is]->template input_channel<0>()), ...);
}(std::make_index_sequence<W>{});
fan->start();
for (auto& n : nodes) n->start();
std::array<std::thread, W> readers;
std::atomic<sclock::time_point> t1;
std::atomic<int> readers_done{0};
for (std::size_t w = 0; w < W; ++w) {
readers[w] = std::thread([&, w] {
for (int i = 0; i < N; ++i) sink_chs[w]->pop();
if (readers_done.fetch_add(1, std::memory_order_acq_rel) + 1
== static_cast<int>(W))
t1.store(sclock::now(), std::memory_order_release);
});
}
auto t0 = sclock::now();
std::thread pusher([&] {
for (int i = 0; i < N; ++i) push_retry(*src_ch, i);
});
pusher.join();
for (auto& r : readers) r.join();
fan->stop();
for (auto& n : nodes) n->stop();
double elapsed = std::chrono::duration<double>(
t1.load(std::memory_order_acquire) - t0).count();
double wus = (elapsed * 1e6) / N - static_cast<double>(work_us);
return {"wide", static_cast<int>(W), work_us, 0, N / elapsed, wus};
}
template<std::size_t W>
static Result bench_wide_pool(int work_us, int pool_threads) {
const int N = items_for(work_us);
const int CAP = N;
auto pool = std::make_shared<ThreadPool>(pool_threads);
auto src_ch = std::make_shared<Channel<int>>(CAP);
auto fan = std::make_unique<FanoutNode<int, W>>(CAP);
fan->template set_input_channel<0>(src_ch);
std::array<std::unique_ptr<PoolChainNode>, W> nodes;
std::array<std::shared_ptr<Channel<int>>, W> sink_chs;
for (std::size_t i = 0; i < W; ++i) {
nodes[i] = std::make_unique<PoolChainNode>(pool, CAP);
sink_chs[i] = std::make_shared<Channel<int>>(CAP);
nodes[i]->template set_output_channel<0>(sink_chs[i].get());
}
[&]<std::size_t... Is>(std::index_sequence<Is...>) {
(fan->template set_output_channel<Is>(
&nodes[Is]->template input_channel<0>()), ...);
}(std::make_index_sequence<W>{});
fan->start();
pool->start();
for (auto& n : nodes) n->start();
std::array<std::thread, W> readers;
std::atomic<sclock::time_point> t1;
std::atomic<int> readers_done{0};
for (std::size_t w = 0; w < W; ++w) {
readers[w] = std::thread([&, w] {
for (int i = 0; i < N; ++i) sink_chs[w]->pop();
if (readers_done.fetch_add(1, std::memory_order_acq_rel) + 1
== static_cast<int>(W))
t1.store(sclock::now(), std::memory_order_release);
});
}
auto t0 = sclock::now();
std::thread pusher([&] {
for (int i = 0; i < N; ++i) push_retry(*src_ch, i);
});
pusher.join();
for (auto& r : readers) r.join();
fan->stop();
for (auto& n : nodes) n->stop();
pool->stop();
double elapsed = std::chrono::duration<double>(
t1.load(std::memory_order_acquire) - t0).count();
double wus = (elapsed * 1e6) / N - static_cast<double>(work_us);
return {"wide", static_cast<int>(W), work_us, pool_threads, N / elapsed, wus};
}
// ── diamond ───────────────────────────────────────────────────────────────────
static Result bench_diamond(int work_us) {
const int N = items_for(work_us, 2);
const int CAP = N;
auto src_ch = std::make_shared<Channel<int>>(CAP);
auto fan = std::make_unique<FanoutNode<int, 2>>(CAP);
fan->template set_input_channel<0>(src_ch);
auto nL = std::make_unique<ChainNode>(CAP);
auto nR = std::make_unique<ChainNode>(CAP);
auto nL2 = std::make_unique<ChainNode>(CAP);
auto nR2 = std::make_unique<ChainNode>(CAP);
auto chL = std::make_shared<Channel<int>>(CAP);
auto chR = std::make_shared<Channel<int>>(CAP);
auto snkL = std::make_shared<Channel<int>>(CAP);
auto snkR = std::make_shared<Channel<int>>(CAP);
fan->template set_output_channel<0>(&nL->template input_channel<0>());
fan->template set_output_channel<1>(&nR->template input_channel<0>());
nL->set_output_channel<0>(chL.get());
nR->set_output_channel<0>(chR.get());
nL2->set_input_channel<0>(chL);
nR2->set_input_channel<0>(chR);
nL2->set_output_channel<0>(snkL.get());
nR2->set_output_channel<0>(snkR.get());
fan->start(); nL->start(); nR->start(); nL2->start(); nR2->start();
std::atomic<sclock::time_point> t1;
std::atomic<int> done{0};
auto make_reader = [&](Channel<int>& ch) {
return std::thread([&] {
for (int i = 0; i < N; ++i) ch.pop();
if (done.fetch_add(1, std::memory_order_acq_rel) + 1 == 2)
t1.store(sclock::now(), std::memory_order_release);
});
};
auto rL = make_reader(*snkL);
auto rR = make_reader(*snkR);
auto t0 = sclock::now();
std::thread pusher([&] {
for (int i = 0; i < N; ++i) push_retry(*src_ch, i);
});
pusher.join(); rL.join(); rR.join();
fan->stop(); nL->stop(); nR->stop(); nL2->stop(); nR2->stop();
double elapsed = std::chrono::duration<double>(
t1.load(std::memory_order_acquire) - t0).count();
double wus = (elapsed * 1e6) / N - static_cast<double>(work_us);
return {"diamond", 4, work_us, 0, N / elapsed, wus};
}
static Result bench_diamond_pool(int work_us, int pool_threads) {
const int N = items_for(work_us, 2);
const int CAP = N;
auto pool = std::make_shared<ThreadPool>(pool_threads);
auto src_ch = std::make_shared<Channel<int>>(CAP);
auto fan = std::make_unique<FanoutNode<int, 2>>(CAP);
fan->template set_input_channel<0>(src_ch);
auto nL = std::make_unique<PoolChainNode>(pool, CAP);
auto nR = std::make_unique<PoolChainNode>(pool, CAP);
auto nL2 = std::make_unique<PoolChainNode>(pool, CAP);
auto nR2 = std::make_unique<PoolChainNode>(pool, CAP);
auto chL = std::make_shared<Channel<int>>(CAP);
auto chR = std::make_shared<Channel<int>>(CAP);
auto snkL = std::make_shared<Channel<int>>(CAP);
auto snkR = std::make_shared<Channel<int>>(CAP);
fan->template set_output_channel<0>(&nL->template input_channel<0>());
fan->template set_output_channel<1>(&nR->template input_channel<0>());
nL->set_output_channel<0>(chL.get());
nR->set_output_channel<0>(chR.get());
nL2->set_input_channel<0>(chL);
nR2->set_input_channel<0>(chR);
nL2->set_output_channel<0>(snkL.get());
nR2->set_output_channel<0>(snkR.get());
fan->start();
pool->start();
nL->start(); nR->start(); nL2->start(); nR2->start();
std::atomic<sclock::time_point> t1;
std::atomic<int> done{0};
auto make_reader = [&](Channel<int>& ch) {
return std::thread([&] {
for (int i = 0; i < N; ++i) ch.pop();
if (done.fetch_add(1, std::memory_order_acq_rel) + 1 == 2)
t1.store(sclock::now(), std::memory_order_release);
});
};
auto rL = make_reader(*snkL);
auto rR = make_reader(*snkR);
auto t0 = sclock::now();
std::thread pusher([&] {
for (int i = 0; i < N; ++i) push_retry(*src_ch, i);
});
pusher.join(); rL.join(); rR.join();
fan->stop();
nL->stop(); nR->stop(); nL2->stop(); nR2->stop();
pool->stop();
double elapsed = std::chrono::duration<double>(
t1.load(std::memory_order_acquire) - t0).count();
double wus = (elapsed * 1e6) / N - static_cast<double>(work_us);
return {"diamond", 4, work_us, pool_threads, N / elapsed, wus};
}
// ── TBB flow graph ────────────────────────────────────────────────────────────
#ifdef KPN_BENCH_TBB
static Result bench_chain_tbb(int depth, int work_us) {
const int N = items_for(work_us, depth);
tbb_flow::graph g;
using FN = tbb_flow::function_node<int, int>;
std::vector<std::unique_ptr<FN>> nodes;
nodes.reserve(depth);
for (int i = 0; i < depth; ++i)
nodes.push_back(std::make_unique<FN>(g, tbb_flow::serial,
[](int x) -> int { return chain_fn(x); }));
for (int i = 0; i + 1 < depth; ++i)
tbb_flow::make_edge(*nodes[i], *nodes[i + 1]);
auto t0 = sclock::now();
for (int i = 0; i < N; ++i) nodes[0]->try_put(i);
g.wait_for_all();
auto t1 = sclock::now();
double elapsed = std::chrono::duration<double>(t1 - t0).count();
double pipeline_us = static_cast<double>(work_us) * (N + depth - 1);
double wus = (elapsed * 1e6 - pipeline_us) / N;
return {"chain_tbb", depth, work_us, -1, N / elapsed, wus};
}
template<std::size_t W>
static Result bench_wide_tbb(int work_us) {
const int N = items_for(work_us);
tbb_flow::graph g;
tbb_flow::broadcast_node<int> fan(g);
using FN = tbb_flow::function_node<int, int>;
std::array<std::unique_ptr<FN>, W> nodes;
for (auto& n : nodes) {
n = std::make_unique<FN>(g, tbb_flow::serial,
[](int x) -> int { return chain_fn(x); });
tbb_flow::make_edge(fan, *n);
}
auto t0 = sclock::now();
for (int i = 0; i < N; ++i) fan.try_put(i);
g.wait_for_all();
auto t1 = sclock::now();
double elapsed = std::chrono::duration<double>(t1 - t0).count();
double wus = (elapsed * 1e6) / N - static_cast<double>(work_us);
return {"wide_tbb", static_cast<int>(W), work_us, -1, N / elapsed, wus};
}
static Result bench_diamond_tbb(int work_us) {
const int N = items_for(work_us, 2);
tbb_flow::graph g;
tbb_flow::broadcast_node<int> fan(g);
using FN = tbb_flow::function_node<int, int>;
auto fn = [](int x) -> int { return chain_fn(x); };
FN nL(g, tbb_flow::serial, fn), nR(g, tbb_flow::serial, fn);
FN nL2(g, tbb_flow::serial, fn), nR2(g, tbb_flow::serial, fn);
tbb_flow::make_edge(fan, nL); tbb_flow::make_edge(fan, nR);
tbb_flow::make_edge(nL, nL2); tbb_flow::make_edge(nR, nR2);
auto t0 = sclock::now();
for (int i = 0; i < N; ++i) fan.try_put(i);
g.wait_for_all();
auto t1 = sclock::now();
double elapsed = std::chrono::duration<double>(t1 - t0).count();
double wus = (elapsed * 1e6) / N - static_cast<double>(work_us);
return {"diamond_tbb", 4, work_us, -1, N / elapsed, wus};
}
#endif // KPN_BENCH_TBB
// ── main ──────────────────────────────────────────────────────────────────────
int main() {
const int work_amts[] = {10, 100, 1000};
const int pool_sizes[] = {1, 2, 4};
std::fprintf(stderr, "%-12s %-8s %-10s %-8s %-18s %-20s\n",
"topology", "size", "work_us", "threads", "items/sec", "overhead_us/item");
std::fprintf(stderr, "%s\n", std::string(78, '-').c_str());
std::printf("topology,size,work_us,threads,items_per_sec,overhead_us_per_item\n");
auto emit = [](const Result& r) {
std::string sched = r.threads < 0 ? "tbb"
: r.threads == 0 ? "priv"
: std::to_string(r.threads);
std::fprintf(stderr, "%-12s %-8d %-10d %-8s %-18.0f %-20.1f\n",
r.topology, r.size, r.work_us, sched.c_str(),
r.items_per_sec, r.overhead_us);
std::printf("%s,%d,%d,%s,%.0f,%.2f\n",
r.topology, r.size, r.work_us, sched.c_str(),
r.items_per_sec, r.overhead_us);
std::fflush(stdout);
};
for (int w : work_amts) {
g_work_us.store(w, std::memory_order_relaxed);
std::fprintf(stderr, "\n── work_us=%-4d private pools ───────────────────────────────────────\n", w);
for (int d : {1, 2, 4, 8, 16, 32}) emit(bench_chain(d, w));
emit(bench_wide<1>(w));
emit(bench_wide<2>(w));
emit(bench_wide<3>(w));
emit(bench_wide<4>(w));
emit(bench_diamond(w));
for (int pt : pool_sizes) {
std::fprintf(stderr, "\n── work_us=%-4d shared pool (%d thread%s) ─────────────────────────────\n",
w, pt, pt == 1 ? "" : "s");
for (int d : {1, 2, 4, 8, 16, 32}) emit(bench_chain_pool(d, w, pt));
emit(bench_wide_pool<1>(w, pt));
emit(bench_wide_pool<2>(w, pt));
emit(bench_wide_pool<3>(w, pt));
emit(bench_wide_pool<4>(w, pt));
emit(bench_diamond_pool(w, pt));
}
#ifdef KPN_BENCH_TBB
std::fprintf(stderr, "\n── work_us=%-4d TBB flow graph ──────────────────────────────────────\n", w);
for (int d : {1, 2, 4, 8, 16, 32}) emit(bench_chain_tbb(d, w));
emit(bench_wide_tbb<1>(w));
emit(bench_wide_tbb<2>(w));
emit(bench_wide_tbb<3>(w));
emit(bench_wide_tbb<4>(w));
emit(bench_diamond_tbb(w));
#endif
}
}
-1073
View File
File diff suppressed because it is too large Load Diff
+55
View File
@@ -0,0 +1,55 @@
# Channels
A `Channel<T>` is a lock-free SPSC (single-producer, single-consumer) ring buffer with atomic wait/notify.
## Semantics
- **Bounded**: fixed capacity set at construction. Default is 5 items.
- **Backpressure**: when full, `push()` throws `ChannelOverflowError` immediately — no blocking, no spin.
- **Blocking consumer**: `pop()` blocks until an item is available or the channel is disabled.
- **Disable**: `channel.disable()` stops accepting pushes and unblocks any waiting `pop()` with `ChannelClosedError`.
## Storage policy
Small trivially-copyable types (≤ 8 bytes) are stored by value. Larger types are heap-allocated and passed via `shared_ptr<const T>` — one allocation per push, zero-copy fan-out:
```cpp
--8<-- "examples/04_storage_policy/main.cpp:storage_policy_spec"
```
Specialize `kpn::ChannelDataSize<T>` for accurate bandwidth reporting on heap-owning types:
```cpp
template<>
struct kpn::ChannelDataSize<cv::Mat> {
static std::size_t bytes(const cv::Mat& m) { return m.total() * m.elemSize(); }
};
```
## Named ports
`in<"name">` and `out<"name">` tag nodes for readable wiring:
```cpp
--8<-- "examples/02_named_ports/main.cpp:named_port_creation"
```
Named ports are checked at compile time — a typo in a port name is a compile error.
## Capacity tuning
Set capacity per node at construction:
```cpp
auto node = make_node<my_func>(/*capacity=*/20);
```
Capacity is rounded up internally to the next power of two. Monitor fill levels via diagnostics to tune for your workload — a too-small capacity causes overflows; a too-large one wastes memory and hides producer/consumer speed mismatches.
## Spin count
`Channel` spins for up to ~4 µs (200 `pause` hints at ~20 ns each on x86) before sleeping on a futex. Set to 0 for power-constrained or predominantly-idle pipelines:
```cpp
Channel<int> ch(/*capacity=*/5, /*spin_count=*/0);
```
+84
View File
@@ -0,0 +1,84 @@
# Error Handling & Events
KPN++ provides three complementary layers for observing and reacting to failures.
---
## 1. Per-node error handler
Called when a node's function throws an unhandled exception. Return `true` to skip the failed invocation and keep running; `false` to stop the node.
```cpp
--8<-- "examples/15_node_error_handler/main.cpp:error_handler"
```
When a node stops (either from `false` return or no handler installed), it:
1. Disables its **input** channels — upstream stops pushing into dead queues.
2. Disables its **output** channels — downstream nodes receive `ChannelClosedError` on their next pop, propagating the shutdown naturally through the graph.
---
## 2. Per-node overflow callback
Fired with a timestamp each time an output push is dropped because the channel is full. The node name is known at registration so it is not included — keeping the callback zero-overhead when unused.
```cpp
--8<-- "examples/16_event_callbacks/main.cpp:per_node_callback"
```
!!! note
The callback is purely informational — the node always continues after an overflow. To stop the node on overflow, call `node.stop()` from inside the callback.
A matching `set_closed_callback()` fires (also with just a timestamp) when the node stops due to a closed upstream channel:
```cpp
node.set_closed_callback([](std::chrono::steady_clock::time_point ts) {
std::cerr << "node stopped at t=" << ts.time_since_epoch().count() << '\n';
});
```
Each node holds two callback slots per event type — one user-set (registered above) and one injected by the network (see below). Both fire independently.
---
## 3. Network-level event handler
One callback for the whole network. Receives the node name (captured in a closure by the network at `build()` / `start()`), a `NodeEvent`, and a timestamp:
```cpp
--8<-- "examples/16_event_callbacks/main.cpp:network_event_handler"
```
`NodeEvent` values:
| Value | Meaning |
|---|---|
| `NodeEvent::Overflow` | An output push was dropped (channel full) |
| `NodeEvent::Closed` | The node stopped (crash or upstream close cascade) |
The network handler and any per-node callbacks are **independent** — both fire when set.
---
## Complete example
`examples/16_event_callbacks/main.cpp` shows a fast producer overflowing a slow consumer, with both a per-node overflow callback and a network-level event handler active simultaneously.
Node functions:
```cpp
--8<-- "examples/16_event_callbacks/main.cpp:node_fns"
```
Per-node overflow callback:
```cpp
--8<-- "examples/16_event_callbacks/main.cpp:per_node_callback"
```
Network-level event handler:
```cpp
--8<-- "examples/16_event_callbacks/main.cpp:network_event_handler"
```
+40
View File
@@ -0,0 +1,40 @@
# Examples
All C++ examples are built by default and registered as CTest smoke tests. Run them all with:
```bash
ctest --test-dir build -L examples
```
## Index
| Example | What it shows |
|---|---|
| `01_hello_pipeline` | Linear pipeline, index-based port wiring |
| `02_named_ports` | `in<>`/`out<>` name tags, named port access |
| `03_multi_output` | Tuple-returning node, per-element routing |
| `04_storage_policy` | `channel_storage_policy` specialisation |
| `05_error_handling` | Diagnostics handler, overflow channel stats |
| `06_watchdog` | Watchdog interval, stall detection |
| `10_static_hello_pipeline` | `StaticNetwork` + `make_network()` |
| `11_static_fanout` | `StaticNetwork` with `FanoutNode` |
| `15_node_error_handler` | `set_error_handler()` — skip or stop on exception |
| `16_event_callbacks` | `set_overflow_callback()`, `set_event_handler()` |
## OpenCV examples (optional)
Built only when OpenCV ≥ 4 is found:
| Example | What it shows |
|---|---|
| `09_opencv_cellshade` | Real-time cell-shading on webcam; `MainThreadNode` for display |
| `12_static_cellshade` | Same pipeline as a `StaticNetwork` |
| `13_debug_cellshade` | Web debug UI overlay on the cell-shading pipeline |
Run the cell-shading example:
```bash
./build/examples/09_opencv_cellshade
# Press 'q' or close the window to stop.
# Falls back to an animated synthetic pattern if no webcam is found.
```
+44
View File
@@ -0,0 +1,44 @@
# Fan-out & Routing
## FanoutNode
Reads one item and pushes a copy to each of N output channels. All downstream nodes receive every item.
```cpp
auto fan = make_fanout<Image, 2>(/*capacity=*/8);
net.connect("src", src.output<0>(), "fan", fan.input<0>())
.connect("fan", fan.output<0>(), "nodeA", nodeA.input<0>())
.connect("fan", fan.output<1>(), "nodeB", nodeB.input<0>());
```
If one downstream channel overflows, that output drops the item independently — the other outputs are unaffected.
See `examples/11_static_fanout`.
## RouterNode
Reads one item and pushes it to exactly one of N outputs, chosen by a selector function:
```cpp
auto router = make_router<Frame, 3>(
[](const Frame& f) -> std::size_t { return f.stream_id % 3; });
net.connect("src", src.output<0>(), "router", router.input<0>())
.connect("router", router.output<0>(), "nodeA", nodeA.input<0>())
.connect("router", router.output<1>(), "nodeB", nodeB.input<0>())
.connect("router", router.output<2>(), "nodeC", nodeC.input<0>());
```
If the selector returns `>= N` the item is silently dropped.
## FilterNode
Reads one item and passes it downstream only when a predicate returns `true`:
```cpp
auto filt = make_filter<Frame>([](const Frame& f) { return f.valid; });
net.connect("src", src.output<0>(), "filt", filt.input<0>())
.connect("filt", filt.output<0>(), "dst", dst.input<0>());
```
+76
View File
@@ -0,0 +1,76 @@
# Getting Started
## Requirements
| Dependency | Version | Notes |
|---|---|---|
| CMake | ≥ 3.21 | |
| C++ compiler | GCC ≥ 11, Clang ≥ 13 | C++20 required |
| nanobind | ≥ 2.1 | auto-fetched; Python ≥ 3.8 |
| Catch2 | v3 | auto-fetched for tests |
| OpenCV | ≥ 4 | optional; only for examples 09/12/13 |
## Build
```bash
cmake -B build # core + tests + C++ examples
cmake --build build --parallel
ctest --test-dir build # run all tests including example smoke tests
```
Enable Python bindings:
```bash
cmake -B build -DKPN_BUILD_PYTHON=ON
cmake --build build --parallel
```
Skip examples:
```bash
cmake -B build -DKPN_BUILD_EXAMPLES=OFF
```
## Your first pipeline
Three functions — source, transform, sink — wired into a `Network`:
```cpp
--8<-- "examples/01_hello_pipeline/main.cpp:basic_node_fns"
```
Create nodes, connect them, build and run:
```cpp
--8<-- "examples/01_hello_pipeline/main.cpp:network_build"
```
That's it. Types are inferred from function signatures. The channel between `src` and `dbl` carries `int`; the channel between `dbl` and `prn` also carries `int`. A type mismatch is a compile error.
## Named ports
For nodes with multiple inputs or outputs, name the ports for clarity:
```cpp
--8<-- "examples/02_named_ports/main.cpp:named_port_creation"
```
Wire by name instead of index:
```cpp
--8<-- "examples/02_named_ports/main.cpp:named_port_network"
```
## Multi-output nodes
Return a `std::tuple` to fan out to multiple downstream nodes:
```cpp
--8<-- "examples/03_multi_output/main.cpp:multi_output_fn"
```
Wire each tuple element to its own downstream node:
```cpp
--8<-- "examples/03_multi_output/main.cpp:fanout_network"
```
+39
View File
@@ -0,0 +1,39 @@
# KPN++
A C++20 [Kahn Process Network](https://en.wikipedia.org/wiki/Kahn_process_networks) library. Each node wraps a plain function and runs concurrently, communicating with downstream nodes via bounded FIFO channels. Includes Python bindings via nanobind.
---
## Why KPN++?
- **Zero boilerplate** — wrap any callable as a node; types flow automatically from the function signature
- **Bounded channels** — backpressure is structural, not bolted on
- **Observable** — per-node and network-level callbacks for overflow and stop events; diagnostics snapshots; optional web UI
- **Composable**`Network` for runtime wiring, `StaticNetwork` for compile-time topology with zero overhead
---
## Quick example
```cpp
#include <kpn/kpn.hpp>
using namespace kpn;
--8<-- "examples/01_hello_pipeline/main.cpp:basic_node_fns"
int main() {
--8<-- "examples/01_hello_pipeline/main.cpp:network_build"
}
```
---
## Install & build
```bash
cmake -B build
cmake --build build --parallel
ctest --test-dir build # unit tests + example smoke tests
```
See [Getting Started](getting-started.md) for full build options.
+67
View File
@@ -0,0 +1,67 @@
# Networks
A `Network` wires nodes together at runtime using a builder chain.
## Building a network
```cpp
--8<-- "examples/01_hello_pipeline/main.cpp:network_build"
```
The builder chain:
| Method | Purpose |
|---|---|
| `.add(name, node)` | Register a node; assigns its name |
| `.connect(src, port, dst, port)` | Wire one output port to one input port |
| `.build()` | Compute topological order; inject network callbacks |
| `.start()` | Start nodes in topological order |
| `.stop()` | Stop all nodes immediately |
| `.shutdown()` | Graceful drain: stop sources first, wait for channels to empty, then stop downstream |
## Port access
Ports are accessed by index or by name:
```cpp
// By index
net.connect("src", src.output<0>(), "dst", dst.input<0>());
// By name (requires named ports)
--8<-- "examples/02_named_ports/main.cpp:named_port_network"
```
## Diagnostics
Install a diagnostics handler to receive periodic snapshots of every node and channel:
```cpp
--8<-- "examples/05_error_handling/main.cpp:diagnostics_handler"
```
Or print a full report at any time:
```cpp
net.print_diagnostics(); // writes to stderr by default
net.print_diagnostics(std::cout);
```
## Network-level event handler
Observe overflow and node-stop events across the entire network in one place:
```cpp
--8<-- "examples/16_event_callbacks/main.cpp:network_event_handler"
```
`NodeEvent` is either `NodeEvent::Overflow` (item dropped on full channel) or `NodeEvent::Closed` (node stopped due to crash or closed upstream channel). See [Error Handling & Events](error-handling.md).
## Shutdown
`net.stop()` halts immediately — all nodes stop in reverse topological order.
`net.shutdown()` drains gracefully: source nodes stop first; their output channels are polled until empty; then the next layer stops, and so on. This ensures no items are lost if downstream nodes are still consuming.
## StaticNetwork
For zero-overhead compile-time topology, see [Static Networks](static-network.md).
+81
View File
@@ -0,0 +1,81 @@
# Nodes
A node wraps any callable. Its input types are inferred from the function's parameter list; its output types from the return type.
## Node types
| Type | Thread model | Use case |
|---|---|---|
| `Node<Func>` | Dedicated thread per node | Default — simplest, most isolated |
| `PoolNode<Func>` | Shared `ThreadPool` | Many nodes, resource-bounded execution |
| `InterruptNode<Func>` | Event-driven, no thread | Camera frame ready, timer tick, socket |
| `FanoutNode<T, N>` | Dedicated thread | Broadcast one item to N outputs |
| `RouterNode<T, N>` | Dedicated thread | Route one item to one of N outputs |
| `FilterNode<T>` | Dedicated thread | Pass items matching a predicate |
## Creating nodes
All node types are created via factory functions that infer types from the callable:
```cpp
// Free function — simplest case
auto node = make_node<my_func>();
// Stateful functor (operator() is the function)
MyProcessor proc;
auto node = make_node(proc);
// Pool node — shares a ThreadPool with other nodes
auto pool = std::make_shared<ThreadPool>(4);
auto node = make_pool_node<my_func>(pool);
// Interrupt node — triggered externally
auto sched = std::make_shared<ThreadPool>(2);
auto node = make_interrupt_node<produce_frame>(sched, out<"frame">{});
camera_sdk.on_frame_ready(node.get_trigger());
```
## Channel capacity
Each node's input FIFO has a configurable capacity (default 5):
```cpp
auto node = make_node<my_func>(/*capacity=*/20);
auto node = make_pool_node<my_func>(pool, /*capacity=*/20);
```
When an upstream push would exceed capacity, `ChannelOverflowError` is thrown and the item is dropped. See [Error Handling & Events](error-handling.md) to observe and react to this.
## Source nodes
A node with no inputs is a source. It self-submits immediately on `start()` and re-submits after each execution:
```cpp
static int produce() {
std::this_thread::sleep_for(std::chrono::milliseconds(10));
return ++counter;
}
auto src = make_node<produce>();
```
!!! tip
Source nodes must sleep or yield to avoid overflowing their output channel. The channel capacity provides the only bound.
## Sink nodes
A node with a `void` return is a sink — it consumes items without producing output:
```cpp
static void print_it(int x) { std::cout << x << '\n'; }
auto snk = make_node<print_it>();
```
## Error handler
When a node's function throws an unhandled exception, the default behaviour is to stop the node (disabling its channels so the shutdown cascades downstream). Install a handler to override:
```cpp
--8<-- "examples/15_node_error_handler/main.cpp:error_handler"
```
See [Error Handling & Events](error-handling.md) for the full picture.
+42
View File
@@ -0,0 +1,42 @@
# Shared Resources
`SharedResource<T>` arbitrates exclusive access to a resource (ONNX session, CUDA stream, serial port) across multiple nodes using a priority-based waiter queue with starvation prevention.
## Usage
```cpp
#include <kpn/shared_resource.hpp>
using namespace kpn;
SharedResource<OnnxSession> model(session_args...);
static cv::Mat run_inference(cv::Mat frame) {
// Acquires the model; releases automatically on scope exit.
auto guard = model.acquire_balanced(in_channel, out_channel);
return guard->Run(frame);
}
```
## Acquire modes
| Method | Priority |
|---|---|
| `acquire()` | Equal (fair FIFO) |
| `acquire(fn)` | Custom — `fn()` returns `float` in `[0, 1]` |
| `acquire_balanced(in_ch, out_ch)` | `input_fill × output_headroom` — highest urgency wins |
`acquire_balanced` favours nodes with full input queues and empty output queues — the node that has the most work to do and nowhere to stall wins the resource next.
## Starvation prevention
Each waiter's effective score grows with elapsed wait time (`0.05` per second by default), ensuring a low-priority node eventually gets served regardless of how frequently higher-priority nodes compete.
## Diagnostics
Register with the network for snapshot reporting:
```cpp
net.register_resource("model", &model);
```
The diagnostics table then shows acquisition count, mean wait time, and current waiter count.
+46
View File
@@ -0,0 +1,46 @@
# Static Networks
`StaticNetwork` encodes the entire topology at compile time using a `make_network()` builder. Nodes and channel types are verified statically with zero runtime overhead.
## Usage
```cpp
#include <kpn/kpn.hpp>
using namespace kpn;
static int produce() { return 42; }
static int double_it(int x) { return x * 2; }
static void print_it(int x) { std::cout << x << '\n'; }
int main() {
auto src = make_node<produce> ();
auto dbl = make_node<double_it>();
auto prn = make_node<print_it> ();
auto net = make_network(
edge(src, src.output<0>(), dbl, dbl.input<0>()),
edge(dbl, dbl.output<0>(), prn, prn.input<0>())
);
net.set_event_handler([](std::string_view name, NodeEvent ev, auto ts) {
// same API as Network
});
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(100));
net.stop();
}
```
See `examples/10_static_hello_pipeline` and `examples/11_static_fanout`.
## When to use
| | `Network` | `StaticNetwork` |
|---|---|---|
| Topology known at | Runtime | Compile time |
| Type checking | Runtime (`dynamic_cast`) | Compile time |
| Overhead | Minimal | Zero |
| Flexibility | Add nodes dynamically | Fixed at compile time |
For most applications `Network` is sufficient. Use `StaticNetwork` when you need the absolute minimum overhead or want compile-time topology verification.
File diff suppressed because it is too large Load Diff
+44
View File
@@ -0,0 +1,44 @@
#include <kpn/kpn.hpp>
#include <iostream>
#include <chrono>
#include <thread>
// A minimal linear pipeline: producer → double → print
//
// [produce] --int--> [double_it] --int--> [print_it]
// --8<-- [start:basic_node_fns]
static int produce() { return 42; }
static int double_it(int x) { return x * 2; }
static void print_it(int x) { std::cout << "result: " << x << '\n'; }
// --8<-- [end:basic_node_fns]
int main() {
using namespace kpn;
// --8<-- [start:index_only_nodes]
auto src = make_node<produce>(5);
auto dbl = make_node<double_it>(5);
auto sink = make_node<print_it>(5);
// --8<-- [end:index_only_nodes]
// Wire channels
auto& dbl_in = dbl.input_channel<0>();
auto& sink_in = sink.input_channel<0>();
src.set_output_channel<0>(&dbl_in);
dbl.set_output_channel<0>(&sink_in);
// --8<-- [start:network_build]
Network net;
net.add("src", src)
.add("dbl", dbl)
.add("sink", sink)
.connect("src", src.output<0>(), "dbl", dbl.input<0>())
.connect("dbl", dbl.output<0>(), "sink", sink.input<0>())
.build();
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(100));
net.stop();
// --8<-- [end:network_build]
}
+82
View File
@@ -0,0 +1,82 @@
// Example 02 — Named Ports
//
// A three-stage text pipeline where port names carry semantic meaning:
//
// [tokenise] --"words"--> [count_words] --"count"--> [report]
// --"words"--> [report]
//
// Named ports let the connect() call read like documentation:
// connect("tok", tok.output<"words">(), "cnt", cnt.input<"words">())
// A typo in the name is a compile-time error, not a runtime surprise.
#include <kpn/kpn.hpp>
#include <chrono>
#include <iostream>
#include <sstream>
#include <string>
#include <thread>
#include <tuple>
#include <vector>
// ── Node functions ────────────────────────────────────────────────────────────
static int sentence_index = 0;
static std::vector<std::string> tokenise() {
static const char* sentences[] = {
"the quick brown fox jumps over the lazy dog",
"kahn process networks are a model of concurrent computation",
"each node runs in its own thread communicating via channels",
"named ports catch wiring mistakes at compile time",
};
std::istringstream ss(sentences[sentence_index++ % 4]);
std::vector<std::string> words;
std::string w;
while (ss >> w) words.push_back(w);
std::this_thread::sleep_for(std::chrono::milliseconds(50));
return words;
}
// Returns (word_count, original_words) — two outputs via tuple
static std::tuple<int, std::vector<std::string>>
count_words(std::vector<std::string> words) {
return {static_cast<int>(words.size()), std::move(words)};
}
static void report(int count, std::vector<std::string> words) {
std::cout << "[" << count << " words] ";
for (auto& w : words) std::cout << w << ' ';
std::cout << '\n';
}
// ── main ──────────────────────────────────────────────────────────────────────
int main() {
using namespace kpn;
// --8<-- [start:named_port_creation]
// tokenise: no inputs, one named output "words"
auto tok = make_node<tokenise>(out<"words">{}, 4);
// count_words: named input "words", named outputs "count" and "words"
auto cnt = make_node<count_words>(in<"words">{}, out<"count", "words">{}, 4);
// report: two named inputs
auto snk = make_node<report>(in<"count", "words">{}, 4);
// --8<-- [end:named_port_creation]
// --8<-- [start:named_port_network]
Network net;
net.add("tok", tok)
.add("cnt", cnt)
.add("snk", snk)
.connect("tok", tok.template output<"words">(), "cnt", cnt.template input<"words">())
.connect("cnt", cnt.template output<"count">(), "snk", snk.template input<"count">())
.connect("cnt", cnt.template output<"words">(), "snk", snk.template input<"words">())
.build();
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(500));
net.stop();
// --8<-- [end:named_port_network]
}
+79
View File
@@ -0,0 +1,79 @@
// Example 03 — Multi-Output (Fan-Out)
//
// A single "parse" node reads "KEY=VALUE" strings and fans out to two
// independent downstream sinks — one for keys, one for values.
//
// +--> [print_key]
// [generate] --string--> [parse]
// +--> [print_value]
//
// The parser returns std::tuple<std::string, std::string>.
// Network::connect() routes each element of the tuple to a different node.
#include <kpn/kpn.hpp>
#include <chrono>
#include <iostream>
#include <string>
#include <thread>
#include <tuple>
// ── Node functions ────────────────────────────────────────────────────────────
static int gen_index = 0;
static std::string generate() {
static const char* pairs[] = {
"host=localhost",
"port=8080",
"timeout=30s",
"retries=3",
"protocol=http2",
};
std::this_thread::sleep_for(std::chrono::milliseconds(60));
return pairs[gen_index++ % 5];
}
// --8<-- [start:multi_output_fn]
// Multi-output: returns (key, value) as a tuple — KPN++ routes each element
// to its own output port automatically.
static std::tuple<std::string, std::string> parse(std::string kv) {
auto sep = kv.find('=');
if (sep == std::string::npos) return {kv, ""};
return {kv.substr(0, sep), kv.substr(sep + 1)};
}
// --8<-- [end:multi_output_fn]
static void print_key(std::string key) {
std::cout << "KEY → " << key << '\n';
}
static void print_value(std::string value) {
std::cout << "VALUE → " << value << '\n';
}
// ── main ──────────────────────────────────────────────────────────────────────
int main() {
using namespace kpn;
// --8<-- [start:fanout_network]
auto gen = make_node<generate>(out<"kv">{}, 4);
auto par = make_node<parse> (in<"kv">{}, out<"key", "value">{}, 4);
auto keys = make_node<print_key> (in<"key">{}, 4);
auto vals = make_node<print_value>(in<"value">{}, 4);
Network net;
net.add("gen", gen)
.add("par", par)
.add("keys", keys)
.add("vals", vals)
.connect("gen", gen.template output<"kv">(), "par", par.template input<"kv">())
.connect("par", par.template output<"key">(), "keys", keys.template input<"key">())
.connect("par", par.template output<"value">(), "vals", vals.template input<"value">())
.build();
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(600));
net.stop();
// --8<-- [end:fanout_network]
}
+120
View File
@@ -0,0 +1,120 @@
// Example 04 — Storage Policy
//
// Demonstrates how KPN++ chooses between by-value and shared_ptr storage
// inside channels depending on the type.
//
// Small trivially-copyable types (int, double, etc.) are stored by value.
// Large or non-trivially-copyable types are stored as shared_ptr<const T>
// — zero copies even across multiple downstream consumers.
//
// This example also shows how to override the policy for a specific type
// with a template specialisation.
//
// [produce_frame] --Frame--> [process_frame] --Frame--> [consume_frame]
// [produce_small] --int----> [double_it] --int----> [print_int]
#include <kpn/kpn.hpp>
#include <array>
#include <chrono>
#include <iostream>
#include <string>
#include <thread>
#include <type_traits>
// ── Types ─────────────────────────────────────────────────────────────────────
// Large frame type — will be stored as shared_ptr<const Frame> by default
struct Frame {
std::array<uint8_t, 4096> pixels{};
int id = 0;
};
// A small struct we force to be stored by value via policy specialisation
struct Tag {
int value = 0;
};
// --8<-- [start:storage_policy_spec]
// Override: store Tag by value despite being a struct
// (it's trivially copyable and small — this just makes the policy explicit)
template<>
struct kpn::channel_storage_policy<Tag> {
static constexpr bool by_value = true;
};
// --8<-- [end:storage_policy_spec]
// ── Node functions ────────────────────────────────────────────────────────────
static int frame_id = 0;
static int tag_id = 0;
static Frame produce_frame() {
std::this_thread::sleep_for(std::chrono::milliseconds(40));
Frame f;
f.id = frame_id++;
f.pixels.fill(static_cast<uint8_t>(f.id & 0xFF));
return f;
}
static Frame process_frame(Frame f) {
// Simulate some work
for (auto& p : f.pixels) p = static_cast<uint8_t>(255 - p);
return f;
}
static void consume_frame(Frame f) {
std::cout << "[frame] id=" << f.id
<< " first_px=" << static_cast<int>(f.pixels[0])
<< " storage=shared_ptr (sizeof Frame = " << sizeof(Frame) << " B)\n";
}
static Tag produce_tag() {
std::this_thread::sleep_for(std::chrono::milliseconds(40));
return {tag_id++};
}
static Tag double_tag(Tag t) { return {t.value * 2}; }
static void print_tag(Tag t) {
std::cout << "[tag] value=" << t.value
<< " storage=by_value (sizeof Tag = " << sizeof(Tag) << " B)\n";
}
// ── main ──────────────────────────────────────────────────────────────────────
int main() {
using namespace kpn;
// Verify storage decisions at compile time
static_assert(!channel_storage_policy<Frame>::by_value,
"Frame should use shared_ptr storage");
static_assert(channel_storage_policy<Tag>::by_value,
"Tag should use by_value storage (via specialisation)");
static_assert(channel_storage_policy<int>::by_value,
"int should use by_value storage");
auto prod_f = make_node<produce_frame> (out<"frame">{}, 4);
auto proc_f = make_node<process_frame> (in<"frame">{}, out<"frame">{}, 4);
auto cons_f = make_node<consume_frame> (in<"frame">{}, 4);
auto prod_t = make_node<produce_tag> (out<"tag">{}, 4);
auto dbl_t = make_node<double_tag> (in<"tag">{}, out<"tag">{}, 4);
auto print_t = make_node<print_tag> (in<"tag">{}, 4);
Network net;
net.add("prod_f", prod_f)
.add("proc_f", proc_f)
.add("cons_f", cons_f)
.add("prod_t", prod_t)
.add("dbl_t", dbl_t)
.add("print_t", print_t)
.connect("prod_f", prod_f.template output<"frame">(), "proc_f", proc_f.template input<"frame">())
.connect("proc_f", proc_f.template output<"frame">(), "cons_f", cons_f.template input<"frame">())
.connect("prod_t", prod_t.template output<"tag">(), "dbl_t", dbl_t.template input<"tag">())
.connect("dbl_t", dbl_t.template output<"tag">(), "print_t", print_t.template input<"tag">())
.build();
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(400));
net.stop();
}
+80
View File
@@ -0,0 +1,80 @@
// Example 05 — Error Handling & Diagnostics
//
// Demonstrates observable failure modes and the diagnostics system:
//
// 1. ChannelOverflowError — a fast producer saturates a slow consumer.
// When the channel is full, push() throws ChannelOverflowError.
// The node's run_loop catches it and prints to stderr.
// Channel statistics (overflows, peak fill) accumulate for the report.
//
// 2. Custom diagnostics handler — instead of the default periodic table,
// install a handler that surfaces only the metrics you care about.
//
// 3. net.print_diagnostics() — print a full report at any time.
//
// Pipeline: [producer] --int--> [slow_consumer]
#include <kpn/kpn.hpp>
#include <chrono>
#include <iostream>
#include <thread>
// ── Node functions ────────────────────────────────────────────────────────────
// Produces at ~100/s — faster than the consumer can keep up (50 ms each)
static int producer() {
std::this_thread::sleep_for(std::chrono::milliseconds(10));
static int n = 0;
return ++n;
}
// Slow consumer: 50 ms per item — will cause channel to fill and overflow
static void slow_consumer(int x) {
std::this_thread::sleep_for(std::chrono::milliseconds(50));
std::cout << "[consumed] " << x << '\n';
}
// ── main ──────────────────────────────────────────────────────────────────────
int main() {
using namespace kpn;
// Capacity=4: fills up quickly when producer outpaces consumer 5:1
auto prod = make_node<producer> (out<"v">{}, /*capacity=*/4);
auto cons = make_node<slow_consumer> (in<"v">{}, /*capacity=*/4);
Network net;
// --8<-- [start:diagnostics_handler]
// Custom diagnostics handler — fires on the watchdog interval.
// Print a concise one-liner rather than the full table.
net.set_diagnostics_handler([](const std::vector<NodeSnapshot>& nodes,
const std::vector<ChannelSnapshot>& channels) {
std::cout << "[diag] ";
for (auto& n : nodes)
std::cout << n.name << "=" << n.throughput_fps << "fps ";
for (auto& c : channels)
std::cout << "channel fill=" << static_cast<int>(c.fill_pct()) << "% "
<< "overflows=" << c.overflows;
std::cout << '\n';
});
// --8<-- [end:diagnostics_handler]
net.set_watchdog_interval(std::chrono::milliseconds(200));
net.add("prod", prod)
.add("cons", cons)
.connect("prod", prod.template output<"v">(), "cons", cons.template input<"v">())
.build();
std::cout << "producer: 10ms/item, consumer: 50ms/item, capacity=4\n"
<< "Overflow messages appear on stderr; diagnostics on stdout.\n\n";
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(800));
net.stop();
// Full report after shutdown — overflow and drop counts are preserved
std::cout << '\n';
net.print_diagnostics();
}
+90
View File
@@ -0,0 +1,90 @@
// Example 06 — Watchdog & Diagnostics (+ optional web debug UI)
//
// A four-stage pipeline with deliberately uneven node speeds:
//
// [source] --int--> [fast_filter] --int--> [slow_transform] --int--> [sink]
//
// The watchdog fires every second and prints a full diagnostics report:
// - frames processed, throughput fps, exec time, blocked time per node
// - channel fill %, peak fill %, pushes, overflows, bandwidth
// - bottleneck hint (node with highest avg exec time)
//
// "slow_transform" sleeps 30 ms per item, making it the obvious bottleneck.
// Watch the channel upstream of it saturate and the fps converge to ~33.
//
// Build with -DKPN_WEB_DEBUG=ON to also get a live D3 graph at localhost:9090.
#include <kpn/kpn.hpp>
#include <chrono>
#include <cmath>
#include <iostream>
#include <thread>
// ── Node functions ────────────────────────────────────────────────────────────
static int source() {
// Produces at ~50 fps — faster than slow_transform (33 fps) so the
// channel between filter and slow gradually fills, but not catastrophically
std::this_thread::sleep_for(std::chrono::milliseconds(20));
static int n = 0;
return ++n;
}
static int fast_filter(int x) {
// Trivial work: ~0.1 ms
return (x % 2 == 0) ? x : x + 1;
}
static int slow_transform(int x) {
// Simulates expensive processing (e.g. a neural net inference step)
std::this_thread::sleep_for(std::chrono::milliseconds(30));
return x * x;
}
static void sink(int x) {
// Print every 10th result to avoid flooding the terminal
if (x % 100 < 4)
std::cout << "[sink] " << x << '\n';
}
// ── main ──────────────────────────────────────────────────────────────────────
int main() {
using namespace kpn;
// Larger capacity buffers so the fast nodes don't immediately overflow
auto src = make_node<source> (out<"v">{}, 16);
auto filt = make_node<fast_filter> (in<"v">{}, out<"v">{}, 16);
auto slow = make_node<slow_transform> (in<"v">{}, out<"v">{}, 8);
auto snk = make_node<sink> (in<"v">{}, 8);
Network net;
// Watchdog fires every 1 second — prints the built-in diagnostics table
// including the "Bottleneck hint" line
net.set_watchdog_interval(std::chrono::milliseconds(1000));
net.add("source", src)
.add("filter", filt)
.add("slow", slow)
.add("sink", snk)
.connect("source", src.template output<"v">(), "filter", filt.template input<"v">())
.connect("filter", filt.template output<"v">(), "slow", slow.template input<"v">())
.connect("slow", slow.template output<"v">(), "sink", snk.template input<"v">())
.build();
std::cout << "Running for 30 seconds — watch 'slow' become the bottleneck.\n"
<< "Watchdog diagnostics print every 1 second.\n";
#ifdef KPN_WEB_DEBUG
net.set_web_debug_port(9090);
std::cout << "Web debug UI: http://localhost:9090 (live graph, auto-updates every 500 ms)\n";
#endif
std::cout << '\n';
net.start();
std::this_thread::sleep_for(std::chrono::seconds(30));
net.stop();
std::cout << "\n=== Final diagnostics ===\n";
net.print_diagnostics();
}
+37
View File
@@ -0,0 +1,37 @@
"""
07_python_network hello pipeline with a Python node in the middle.
Graph:
[ProduceNode] --int--> [py_double] --int--> [PrintItNode]
ProduceNode and PrintItNode are C++ nodes wrapped in VariantNodeWrapper.
py_double is a pure Python callable doubles its input using Python arithmetic.
"""
import sys
import time
sys.path.insert(0, "build/python")
import kpn_python as kpn
def py_double(x: int) -> int:
return x * 2
net = kpn.Network()
net.add("src", kpn.make_produce())
net.add_node("dbl", py_double, inputs=["int"], outputs=["int"])
net.add("sink", kpn.make_print_it())
net.connect("src", 0, "dbl", 0)
net.connect("dbl", 0, "sink", 0)
net.build()
net.start()
time.sleep(0.1)
net.stop()
# Drop the network deterministically: it holds the Python callable, which forms
# a reference cycle via globals(). Deleting the global breaks it so the network
# is reclaimed now rather than lingering to interpreter shutdown.
del net
+55
View File
@@ -0,0 +1,55 @@
"""
08_python_subport drive a *Python* node from Python via write()/read() taps.
Graph:
(fed by net.write()) --int--> [py_triple] --int--> (tapped by net.read())
Unlike 07, there is no C++ source or sink here: the only node in the network is
a pure-Python function, py_triple. Python plays *both* the producer and the
consumer by using the subport taps:
* net.write("py", 0, v) injects v into py_triple's input (Python -> network)
* net.read("py", 0) pulls py_triple's output back out (network -> Python)
This closes the loop the old version left as a "#todo": a value flows from
Python, through a Python node running inside the network, and back to Python.
"""
import sys
sys.path.insert(0, "build/python") # for `python examples/.../example.py` from repo root
import kpn_python as kpn
def py_triple(x: int) -> int:
return x * 3
net = kpn.Network()
# The whole network is a single Python node with a tapped input and output.
net.add_node("py", py_triple, inputs=["int"], outputs=["int"])
net.build()
net.start()
# Push values in from Python and read the Python node's results back out.
inputs = [1, 2, 7, 10, 100]
results = []
for v in inputs:
net.write("py", 0, v) # Python -> py_triple input
results.append(net.read("py", 0)) # py_triple output -> Python
net.stop()
print("inputs written from Python: ", inputs)
print("outputs read from py_triple:", results)
expected = [v * 3 for v in inputs]
assert results == expected, f"expected {expected}, got {results}"
print("all correct (x * 3 computed by a Python node inside the network)")
# Drop the network deterministically. The network holds the Python callable,
# which (via globals) forms a reference cycle; deleting the global breaks it so
# the network is reclaimed promptly instead of lingering to interpreter exit.
del net
@@ -0,0 +1,53 @@
#include <opencv2/videoio.hpp>
#include <chrono>
#include <algorithm>
#include <numeric>
#include <vector>
#include <cstdio>
static void bench(const char* name, int device, int backend, int W, int H, int frames) {
cv::VideoCapture cap(device, backend);
if (!cap.isOpened()) {
std::printf("%-12s failed to open /dev/video%d\n", name, device);
return;
}
cap.set(cv::CAP_PROP_FRAME_WIDTH, W);
cap.set(cv::CAP_PROP_FRAME_HEIGHT, H);
// Warm up
for (int i = 0; i < 5; ++i) cap.grab();
std::vector<double> times;
times.reserve(frames);
for (int i = 0; i < frames; ++i) {
auto t0 = std::chrono::steady_clock::now();
bool ok = cap.grab();
double ms = std::chrono::duration<double, std::milli>(
std::chrono::steady_clock::now() - t0).count();
if (ok) times.push_back(ms);
}
cap.release();
if (times.empty()) {
std::printf("%-12s no frames captured\n", name);
return;
}
std::sort(times.begin(), times.end());
double avg = std::accumulate(times.begin(), times.end(), 0.0) / times.size();
double mn = times.front();
double mx = times.back();
double p95 = times[times.size() * 95 / 100];
std::printf("%-12s avg=%6.1fms min=%5.1fms p95=%6.1fms max=%6.1fms (%zu frames)\n",
name, avg, mn, p95, mx, times.size());
}
int main(int argc, char** argv) {
int device = argc > 1 ? std::atoi(argv[1]) : 0;
int frames = argc > 2 ? std::atoi(argv[2]) : 60;
int W = 1920, H = 1080;
std::printf("Benchmarking /dev/video%d at %dx%d, %d frames each\n\n", device, W, H, frames);
bench("V4L2", device, cv::CAP_V4L2, W, H, frames);
bench("GStreamer", device, cv::CAP_GSTREAMER, W, H, frames);
return 0;
}
@@ -0,0 +1,101 @@
"""
09_opencv_cellshade/example_hybrid.py
Hybrid cell-shading pipeline: C++ nodes handle capture, grayscale conversion,
and edge detection; a Python/numpy function replaces the C++ quantise node;
Python drives the display loop using cv2.
Pipeline:
[py_quantise]
[CaptureNode] [CompositeNode]result cv2.imshow
out0=colour [ToGrayNode][EdgesNode] edges cv2.imshow
out1=grey
For a pure-C++ version see main.cpp; for the C++ static-network version see
12_static_cellshade/main.cpp.
Press 'q' or Esc to stop.
"""
import sys
import os
# Adjust path to wherever CMake placed the .so
BUILD_DIR = os.environ.get("KPN_BUILD_DIR",
os.path.join(os.path.dirname(__file__),
"../../build/examples"))
sys.path.insert(0, BUILD_DIR)
import numpy as np
import cv2
import kpn_opencv as kpn
# ── Python node: replace the C++ quantise with numpy ─────────────────────────
# Receives and returns a BGR numpy array (H×W×3 uint8).
def py_quantise(bgr: np.ndarray) -> np.ndarray:
levels = 4
step = 256 // levels
q = (bgr.astype(np.int32) // step) * step + (step // 2)
return q.clip(0, 255).astype(np.uint8)
# ── Build network ─────────────────────────────────────────────────────────────
net = kpn.Network()
net.add("src", kpn.make_capture()) # out0=colour, out1=grey
net.add_node("quant", py_quantise, # Python node — numpy in/out
inputs=["mat"], outputs=["mat"])
net.add("gray", kpn.make_to_gray()) # in0=bgr → out0=gray
net.add("edges", kpn.make_edges()) # in0=gray → out0=edge_mask
net.add("comp", kpn.make_composite()) # in0=edge_mask, in1=colour
# out0=result, out1=edge_mask
# src.colour → py_quantise
net.connect("src", 0, "quant", 0)
# src.grey → to_gray
net.connect("src", 1, "gray", 0)
# gray → edges
net.connect("gray", 0, "edges", 0)
# quantised colour → composite.colour (input slot 1)
net.connect("quant", 0, "comp", 1)
# edge mask → composite.edges (input slot 0)
net.connect("edges", 0, "comp", 0)
net.build()
net.start()
# ── Display loop (drives GUI on this thread) ──────────────────────────────────
cv2.namedWindow("Cell Shade (Python quant)", cv2.WINDOW_NORMAL)
cv2.namedWindow("Edge Mask", cv2.WINDOW_NORMAL)
cv2.resizeWindow("Cell Shade (Python quant)", 1280, 720)
cv2.resizeWindow("Edge Mask", 640, 360)
try:
while True:
# Blocking reads — GIL released while waiting so C++ threads can run
result = net.read("comp", 0) # composite frame (BGR numpy array)
edges = net.read("comp", 1) # edge mask (grayscale numpy array)
cv2.imshow("Cell Shade (Python quant)", result)
cv2.imshow("Edge Mask", cv2.cvtColor(edges, cv2.COLOR_GRAY2BGR))
key = cv2.waitKey(1)
if key in (ord('q'), 27):
break
# Check windows still open
try:
if cv2.getWindowProperty("Cell Shade (Python quant)",
cv2.WND_PROP_VISIBLE) < 1:
break
except cv2.error:
break
finally:
net.stop()
cv2.destroyAllWindows()
del net # let C++ destructor run before nanobind tears down
+177
View File
@@ -0,0 +1,177 @@
#define KPN_BUILD_PYTHON
#include <kpn/python/auto_bind.hpp>
#include <nanobind/ndarray.h>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/videoio.hpp>
#include <chrono>
#include <cmath>
#include <iostream>
#include <thread>
#include <tuple>
namespace nb = nanobind;
using namespace kpn;
using namespace kpn::python;
// ── PythonConverter<cv::Mat> ──────────────────────────────────────────────────
// Converts cv::Mat ↔ numpy array (uint8, HxW or HxWxC shape).
//
// to_python: clones the mat onto the heap; the numpy array owns it via a
// capsule deleter — no shared cv::Mat refcount dangling after the Variant dies.
// from_python: calls numpy.ascontiguousarray, then clones into an owned cv::Mat.
namespace kpn {
template<> struct PythonConverter<cv::Mat> {
static constexpr const char* type_name = "mat";
static nb::object to_python(const cv::Mat& m) {
// Must be called with the GIL held (always true: called from read() or
// from within the gil_scoped_acquire block in PyNode::run_loop).
auto np = nb::module_::import_("numpy");
cv::Mat c = m.clone(); // ensure contiguous, independently owned
nb::bytes raw(reinterpret_cast<const char*>(c.data),
c.total() * c.elemSize());
nb::object arr = np.attr("frombuffer")(raw, "uint8");
int H = c.rows, W = c.cols, C = c.channels();
arr = arr.attr("reshape")(
C > 1 ? nb::make_tuple(H, W, C) : nb::make_tuple(H, W));
return arr.attr("copy")(); // writable, lifetime-independent copy
}
static cv::Mat from_python(nb::object o) {
auto np = nb::module_::import_("numpy");
// Ensure contiguous uint8 layout (in-place if already compatible)
nb::object arr = np.attr("ascontiguousarray")(o, "uint8");
auto shape = nb::cast<std::vector<int>>(arr.attr("shape"));
if (shape.size() < 2 || shape.size() > 3)
throw std::runtime_error(
"cv::Mat from_python: expected 2D (H×W) or 3D (H×W×C) uint8 array");
int H = shape[0], W = shape[1];
int C = (shape.size() == 3) ? shape[2] : 1;
int type = C > 1 ? CV_8UC(C) : CV_8UC1;
// Cast to ndarray to get the raw data pointer
auto binfo = nb::cast<nb::ndarray<nb::numpy, uint8_t>>(arr);
cv::Mat wrap(H, W, type, binfo.data());
return wrap.clone(); // own the pixel data
}
};
} // namespace kpn
// ── Pipeline functions ────────────────────────────────────────────────────────
static cv::Mat make_gradient(int W, int H) {
cv::Mat xr(H, W, CV_8UC1), yg(H, W, CV_8UC1), b(H, W, CV_8UC1, cv::Scalar(128));
for (int x = 0; x < W; ++x) xr.col(x).setTo(x * 255 / W);
for (int y = 0; y < H; ++y) yg.row(y).setTo(y * 255 / H);
cv::Mat channels[3] = {b, yg, xr};
cv::Mat grad;
cv::merge(channels, 3, grad);
return grad;
}
static std::tuple<cv::Mat, cv::Mat> capture() {
constexpr int W = 640, H = 480;
static cv::VideoCapture cap;
static bool opened = false;
if (!opened) {
opened = true;
cap.open(0, cv::CAP_V4L2);
if (cap.isOpened()) {
cap.set(cv::CAP_PROP_FRAME_WIDTH, W);
cap.set(cv::CAP_PROP_FRAME_HEIGHT, H);
} else {
std::cerr << "[capture] no webcam — using synthetic animated pattern\n";
}
}
cv::Mat frame;
if (cap.isOpened()) {
auto t0 = std::chrono::steady_clock::now();
cap >> frame;
auto elapsed = std::chrono::steady_clock::now() - t0;
if (elapsed < std::chrono::milliseconds(20))
std::this_thread::sleep_for(std::chrono::milliseconds(33) - elapsed);
if (frame.empty()) frame = cv::Mat::zeros(H, W, CV_8UC3);
} else {
static int tick = 0;
static cv::Mat grad = make_gradient(W, H);
++tick;
frame = grad.clone();
int r = 150 + (tick % 80) * 4;
cv::circle(frame, {W/2, H/2}, r, {255, 200, 0}, -1);
cv::circle(frame, {W/2, H/2}, r / 2, { 0, 128, 255}, -1);
cv::circle(frame, {W*2/5, H*2/5}, r / 3, {200, 0, 200}, -1);
std::this_thread::sleep_for(std::chrono::milliseconds(33));
}
return {frame.clone(), frame.clone()};
}
static cv::Mat to_gray(cv::Mat bgr) {
cv::Mat gray;
cv::cvtColor(bgr, gray, cv::COLOR_BGR2GRAY);
return gray;
}
static cv::Mat edges_fn(cv::Mat gray) {
cv::Mat blurred, mask;
cv::GaussianBlur(gray, blurred, {5, 5}, 0);
cv::Canny(blurred, mask, 50, 150);
return mask;
}
static cv::Mat quantise(cv::Mat bgr) {
constexpr int levels = 4;
constexpr double step = 256.0 / levels;
static const cv::Mat lut = []() {
cv::Mat l(1, 256, CV_8UC1);
for (int i = 0; i < 256; ++i)
l.at<uchar>(i) = cv::saturate_cast<uchar>(
std::floor(i / step) * step + step / 2.0);
return l;
}();
cv::Mat out;
cv::LUT(bgr, lut, out);
return out;
}
// Returns composite frame AND edge mask so the display node can show both
// without needing a fan-out on the edges channel.
static std::tuple<cv::Mat, cv::Mat> composite(cv::Mat edge_mask, cv::Mat colour) {
cv::Mat result = colour.clone();
result.setTo(cv::Scalar(0, 0, 0), edge_mask);
return {result, edge_mask};
}
// ── Registry ──────────────────────────────────────────────────────────────────
// Variant deduced as std::variant<cv::Mat> — every node uses only cv::Mat.
using CvNodes = NodeRegistry<
Entry<capture, "capture">,
Entry<to_gray, "to_gray">,
Entry<edges_fn, "edges">,
Entry<quantise, "quantise">,
Entry<composite, "composite">
>;
// ── Module ────────────────────────────────────────────────────────────────────
NB_MODULE(kpn_opencv, m) {
m.doc() = "KPN++ OpenCV bindings for the cell-shading pipeline";
// Registers: Network, INode, CaptureNode, ToGrayNode, EdgesNode,
// QuantiseNode, CompositeNode, and make_<name>() factories.
// Network.add_node(name, callable, inputs=["mat"], outputs=["mat"])
// accepts Python callables that receive/return numpy uint8 arrays.
bind_network<CvNodes>(m);
// Note: bind_debug is omitted here — cv::Mat functions cannot be called
// directly from Python without the variant/network machinery. Use
// net.write() + net.read() to inject/inspect individual nodes instead.
}
+206
View File
@@ -0,0 +1,206 @@
#include <kpn/kpn.hpp>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio.hpp>
#include <iostream>
#include <tuple>
#include <thread>
#include <chrono>
// Teach KPN how many bytes a cv::Mat actually carries (header + pixel data).
template<>
struct kpn::ChannelDataSize<cv::Mat> {
static std::size_t bytes(const cv::Mat& m) { return m.total() * m.elemSize(); }
};
// ── Cell-shading pipeline ─────────────────────────────────────────────────────
//
// [capture] --"colour"--> [quantise] ──────────────────────────┐
// ├──> [composite] ──"result"──┐
// [capture] --"grey"---> [to_gray] --> [edges] ──"edges"───────┘ │
// └──────────────────────────────"edges"──────────┴──> [display]
//
// DisplayNode derives from MainThreadNode<> — two inputs (composite + raw edges),
// two windows opened in the constructor. step() is called on the main thread.
// ── Gradient base for synthetic pattern ──────────────────────────────────────
static cv::Mat make_gradient(int W, int H) {
cv::Mat xr(H, W, CV_8UC1), yg(H, W, CV_8UC1), b(H, W, CV_8UC1, cv::Scalar(128));
for (int x = 0; x < W; ++x) xr.col(x).setTo(x * 255 / W);
for (int y = 0; y < H; ++y) yg.row(y).setTo(y * 255 / H);
cv::Mat channels[3] = {b, yg, xr};
cv::Mat grad;
cv::merge(channels, 3, grad);
return grad;
}
// ── Pipeline functions ────────────────────────────────────────────────────────
// --8<-- [start:capture_fn]
static std::tuple<cv::Mat, cv::Mat> capture() {
constexpr int W = 640, H = 480;
static cv::VideoCapture cap;
static bool opened = false;
if (!opened) {
opened = true;
cap.open(0, cv::CAP_V4L2);
if (cap.isOpened()) {
cap.set(cv::CAP_PROP_FRAME_WIDTH, W);
cap.set(cv::CAP_PROP_FRAME_HEIGHT, H);
} else {
std::cerr << "[capture] no webcam — using synthetic animated pattern\n";
}
}
cv::Mat frame;
if (cap.isOpened()) {
auto t0 = std::chrono::steady_clock::now();
cap >> frame;
auto elapsed = std::chrono::steady_clock::now() - t0;
if (elapsed < std::chrono::milliseconds(20))
std::this_thread::sleep_for(std::chrono::milliseconds(33) - elapsed);
if (frame.empty()) frame = cv::Mat::zeros(H, W, CV_8UC3);
} else {
static int tick = 0;
static cv::Mat grad = make_gradient(W, H);
++tick;
frame = grad.clone();
int r = 150 + (tick % 80) * 4;
cv::circle(frame, {W/2, H/2}, r, {255, 200, 0}, -1);
cv::circle(frame, {W/2, H/2}, r / 2, { 0, 128, 255}, -1);
cv::circle(frame, {W*2/5, H*2/5}, r / 3, {200, 0, 200}, -1);
std::this_thread::sleep_for(std::chrono::milliseconds(33));
}
return {frame.clone(), frame.clone()};
}
// --8<-- [end:capture_fn]
static cv::Mat to_gray(cv::Mat bgr) {
cv::Mat gray;
cv::cvtColor(bgr, gray, cv::COLOR_BGR2GRAY);
return gray;
}
static cv::Mat edges_fn(cv::Mat gray) {
cv::Mat blurred, mask;
cv::GaussianBlur(gray, blurred, {5, 5}, 0);
cv::Canny(blurred, mask, 50, 150);
return mask;
}
static cv::Mat quantise(cv::Mat bgr) {
constexpr int levels = 4;
constexpr double step = 256.0 / levels;
static const cv::Mat lut = []() {
cv::Mat l(1, 256, CV_8UC1);
for (int i = 0; i < 256; ++i)
l.at<uchar>(i) = cv::saturate_cast<uchar>(
std::floor(i / step) * step + step / 2.0);
return l;
}();
cv::Mat out;
cv::LUT(bgr, lut, out);
return out;
}
// Returns both the composite frame and the original edge mask so downstream
// nodes (display) can receive both without fan-out on the edges channel.
static std::tuple<cv::Mat, cv::Mat> composite(cv::Mat edge_mask, cv::Mat colour) {
cv::Mat result = colour.clone();
result.setTo(cv::Scalar(0, 0, 0), edge_mask);
return {result, edge_mask};
}
// ── DisplayNode ───────────────────────────────────────────────────────────────
//
// Two inputs: the cell-shaded composite frame and the raw edge mask.
// MainThreadNode<> provides channel ownership, INode boilerplate, and stats.
// The constructor opens both windows on the main thread (Wayland requirement).
// operator() is called by step() whenever both channels have a frame ready.
// --8<-- [start:display_node]
class DisplayNode : public kpn::MainThreadNode<DisplayNode,
kpn::in<"composite", "edges">,
cv::Mat, cv::Mat> {
public:
DisplayNode() : MainThreadNode(8) {
cv::namedWindow("Cell Shade", cv::WINDOW_NORMAL);
cv::namedWindow("Edge Mask", cv::WINDOW_NORMAL);
cv::resizeWindow("Cell Shade", 1280, 720);
cv::resizeWindow("Edge Mask", 640, 360);
}
~DisplayNode() { cv::destroyAllWindows(); }
bool operator()(cv::Mat composite, cv::Mat edges) {
cv::imshow("Cell Shade", composite);
cv::Mat edges_bgr;
cv::cvtColor(edges, edges_bgr, cv::COLOR_GRAY2BGR);
cv::imshow("Edge Mask", edges_bgr);
int key = cv::waitKey(1);
if (key == 'q' || key == 27) return false;
return window_open("Cell Shade") && window_open("Edge Mask");
}
private:
static bool window_open(const char* name) {
try { return cv::getWindowProperty(name, cv::WND_PROP_VISIBLE) >= 1; }
catch (const cv::Exception&) { return false; }
}
};
// --8<-- [end:display_node]
// ─────────────────────────────────────────────────────────────────────────────
int main() {
using namespace kpn;
// --8<-- [start:opencv_network]
auto src = make_node<capture> (out<"colour","grey">{}, 8);
auto gray_node = make_node<to_gray> (in<"bgr">{}, out<"gray">{}, 8);
auto edge_node = make_node<edges_fn> (in<"gray">{}, out<"edges">{}, 8);
auto quant = make_node<quantise> (in<"bgr">{}, out<"quantised">{}, 8);
auto comp = make_node<composite>(in<"edges","colour">{}, out<"result","edges">{}, 8);
// DisplayNode: two windows opened in constructor, step() drives main thread.
DisplayNode disp;
Network net;
net.add("src", src)
.add("gray", gray_node)
.add("edges", edge_node)
.add("quant", quant)
.add("comp", comp)
.add("display", disp)
.connect("src", src.template output<"colour">(), "quant", quant.template input<"bgr">())
.connect("quant", quant.template output<"quantised">(), "comp", comp.template input<"colour">())
.connect("src", src.template output<"grey">(), "gray", gray_node.template input<"bgr">())
.connect("gray", gray_node.template output<"gray">(), "edges", edge_node.template input<"gray">())
.connect("edges", edge_node.template output<"edges">(), "comp", comp.template input<"edges">())
.connect("comp", comp.template output<"result">(), "display", disp.template input<"composite">())
.connect("comp", comp.template output<"edges">(), "display", disp.template input<"edges">())
.build();
// --8<-- [end:opencv_network]
net.set_watchdog_interval(std::chrono::milliseconds(5000));
#ifdef KPN_WEB_DEBUG
net.set_web_debug_port(9090);
#endif
std::cout << "Cell-shading pipeline running. Press 'q' to stop.\n";
std::cout << "Web debug UI: http://localhost:9090\n";
// --8<-- [start:main_thread_step]
net.start();
// Main thread drives display — imshow/waitKey stay on the GUI thread.
// step() returns false when operator() returns false (q pressed / window closed).
while (disp.step())
cv::waitKey(8); // yield event loop when no frame ready
net.stop();
// --8<-- [end:main_thread_step]
return 0;
}
@@ -0,0 +1,34 @@
// Example 10 — Static Hello Pipeline
//
// The same linear pipeline as example 01, built with make_network() instead
// of the runtime Network builder. The topology is fully known at compile time:
// cycle detection is a static_assert, no build() step is needed, and start/stop
// require no virtual dispatch through a string-keyed node map.
//
// [produce] --int--> [double_it] --int--> [print_it]
#include <kpn/kpn.hpp>
#include <chrono>
#include <iostream>
#include <thread>
static int produce() { return 42; }
static int double_it(int x) { return x * 2; }
static void print_it(int x) { std::cout << "result: " << x << '\n'; }
int main() {
using namespace kpn;
auto src = make_node<produce, "src" >(5);
auto dbl = make_node<double_it, "dbl" >(5);
auto sink = make_node<print_it, "sink">(5);
auto net = make_network(
edge(src.output<0>(), dbl.input<0>()),
edge(dbl.output<0>(), sink.input<0>())
);
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(100));
net.stop();
}
+55
View File
@@ -0,0 +1,55 @@
// Example 11 — Static Fan-Out (automatic FanoutNode insertion)
//
// Two consumers read from the same output port of [generate]. With the runtime
// Network builder this would require an explicit make_fanout<>. With make_network()
// the duplicate source port is detected at compile time and a FanoutNode<string,2>
// is inserted automatically — the user just writes two edges from the same port.
//
// +--> [print_key]
// [generate] --string--> [fan]
// +--> [print_upper]
//
// The FanoutNode<string,2> is owned by the StaticNetwork and invisible to the user.
#include <kpn/kpn.hpp>
#include <algorithm>
#include <chrono>
#include <iostream>
#include <string>
#include <thread>
static int gen_index = 0;
static std::string generate() {
static const char* words[] = {"hello", "kpn", "fanout", "static", "network"};
std::this_thread::sleep_for(std::chrono::milliseconds(80));
return words[gen_index++ % 5];
}
static void print_lower(std::string s) {
std::cout << "lower: " << s << '\n';
}
static void print_upper(std::string s) {
std::transform(s.begin(), s.end(), s.begin(), ::toupper);
std::cout << "upper: " << s << '\n';
}
int main() {
using namespace kpn;
auto gen = make_node<generate, "gen" >(5);
auto lower = make_node<print_lower, "lower">(5);
auto upper = make_node<print_upper, "upper">(5);
// Two edges from gen.output<0>() — make_network() detects the fan-out
// and inserts FanoutNode<std::string, 2> automatically.
auto net = make_network(
edge(gen.output<0>(), lower.input<0>()),
edge(gen.output<0>(), upper.input<0>())
);
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(500));
net.stop();
}
+217
View File
@@ -0,0 +1,217 @@
// Example 12 — Static Cell-Shading Pipeline with Auto Fan-Out
//
// The same cell-shading effect as example 09, rebuilt with make_network().
//
// Key differences from example 09:
//
// 1. Fan-out is automatic. The edge detector output feeds both the compositing
// node and the debug display window. In example 09 this required composite()
// to re-output the edge mask as a second return value (a workaround). Here
// make_network() detects the duplicate source port and inserts
// FanoutNode<cv::Mat, 2> automatically — composite() is a clean single-output
// node.
//
// 2. No add()/connect()/build() ceremony. The full topology is expressed once
// in the make_network() call. Cycle detection and duplicate-tag checking are
// compile-time static_asserts.
//
// 3. Every node has a Label NTTP so the web debug UI shows real names.
//
// Topology:
//
// [capture] --colour--> [quant] ──────────────────────────────> [comp] --> [display_composite]
// [capture] --grey----> [to_gray] --> [edges] --edges--(fan)--> [comp]
// --edges----------> [display_edges] ← auto-fanout
//
// Note: capture returns std::tuple<cv::Mat, cv::Mat> (colour, grey).
// The two outputs are separate ports routed independently.
#include <kpn/kpn.hpp>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio.hpp>
#include <iostream>
#include <tuple>
#include <thread>
#include <chrono>
// ── Gradient base for synthetic pattern ───────────────────────────────────────
static cv::Mat make_gradient(int W, int H) {
cv::Mat xr(H, W, CV_8UC1), yg(H, W, CV_8UC1), b(H, W, CV_8UC1, cv::Scalar(128));
for (int x = 0; x < W; ++x) xr.col(x).setTo(x * 255 / W);
for (int y = 0; y < H; ++y) yg.row(y).setTo(y * 255 / H);
cv::Mat channels[3] = {b, yg, xr};
cv::Mat grad;
cv::merge(channels, 3, grad);
return grad;
}
// ── Pipeline functions ────────────────────────────────────────────────────────
static std::tuple<cv::Mat, cv::Mat> capture() {
constexpr int W = 640, H = 480;
static cv::VideoCapture cap;
static bool opened = false;
if (!opened) {
opened = true;
cap.open(0, cv::CAP_V4L2);
if (cap.isOpened()) {
cap.set(cv::CAP_PROP_FRAME_WIDTH, W);
cap.set(cv::CAP_PROP_FRAME_HEIGHT, H);
} else {
std::cerr << "[capture] no webcam — using synthetic animated pattern\n";
}
}
cv::Mat frame;
if (cap.isOpened()) {
auto t0 = std::chrono::steady_clock::now();
cap >> frame;
auto elapsed = std::chrono::steady_clock::now() - t0;
if (elapsed < std::chrono::milliseconds(20))
std::this_thread::sleep_for(std::chrono::milliseconds(33) - elapsed);
if (frame.empty()) frame = cv::Mat::zeros(H, W, CV_8UC3);
} else {
static int tick = 0;
static cv::Mat grad = make_gradient(W, H);
++tick;
frame = grad.clone();
int r = 150 + (tick % 80) * 4;
cv::circle(frame, {W/2, H/2}, r, {255, 200, 0}, -1);
cv::circle(frame, {W/2, H/2}, r / 2, { 0, 128, 255}, -1);
cv::circle(frame, {W*2/5, H*2/5}, r / 3, {200, 0, 200}, -1);
std::this_thread::sleep_for(std::chrono::milliseconds(33));
}
return {frame.clone(), frame.clone()};
}
static cv::Mat to_gray(cv::Mat bgr) {
cv::Mat gray;
cv::cvtColor(bgr, gray, cv::COLOR_BGR2GRAY);
return gray;
}
static cv::Mat edges_fn(cv::Mat gray) {
cv::Mat blurred, mask;
cv::GaussianBlur(gray, blurred, {5, 5}, 0);
cv::Canny(blurred, mask, 50, 150);
return mask;
}
static cv::Mat quantise(cv::Mat bgr) {
constexpr int levels = 4;
constexpr double step = 256.0 / levels;
static const cv::Mat lut = []() {
cv::Mat l(1, 256, CV_8UC1);
for (int i = 0; i < 256; ++i)
l.at<uchar>(i) = cv::saturate_cast<uchar>(
std::floor(i / step) * step + step / 2.0);
return l;
}();
cv::Mat out;
cv::LUT(bgr, lut, out);
return out;
}
// Clean single-output composite — no longer needs to pass edges through.
static cv::Mat composite(cv::Mat edge_mask, cv::Mat colour) {
cv::Mat result = colour.clone();
result.setTo(cv::Scalar(0, 0, 0), edge_mask);
return result;
}
// ── Display nodes ─────────────────────────────────────────────────────────────
//
// Two separate MainThreadNode subclasses — one for the composited result,
// one for the raw edge mask. Each runs on the main thread via step().
// The fan-out from [edges] to both consumers is inserted automatically by
// make_network().
class DisplayComposite : public kpn::MainThreadNode<DisplayComposite,
kpn::in<"composite">,
cv::Mat> {
public:
// Label and unique_tag for StaticNetwork identity
static constexpr std::string_view label() { return "display_composite"; }
static constexpr std::size_t unique_tag = 0;
DisplayComposite() : MainThreadNode(8) {
cv::namedWindow("Cell Shade", cv::WINDOW_NORMAL);
cv::resizeWindow("Cell Shade", 1280, 720);
}
~DisplayComposite() { cv::destroyWindow("Cell Shade"); }
bool operator()(cv::Mat frame) {
cv::imshow("Cell Shade", frame);
int key = cv::waitKey(1);
if (key == 'q' || key == 27) return false;
try { return cv::getWindowProperty("Cell Shade", cv::WND_PROP_VISIBLE) >= 1; }
catch (const cv::Exception&) { return false; }
}
};
class DisplayEdges : public kpn::MainThreadNode<DisplayEdges,
kpn::in<"edges">,
cv::Mat> {
public:
static constexpr std::string_view label() { return "display_edges"; }
static constexpr std::size_t unique_tag = 1;
DisplayEdges() : MainThreadNode(8) {
cv::namedWindow("Edge Mask", cv::WINDOW_NORMAL);
cv::resizeWindow("Edge Mask", 640, 360);
}
~DisplayEdges() { cv::destroyWindow("Edge Mask"); }
bool operator()(cv::Mat mask) {
cv::Mat bgr;
cv::cvtColor(mask, bgr, cv::COLOR_GRAY2BGR);
cv::imshow("Edge Mask", bgr);
cv::waitKey(1);
try { return cv::getWindowProperty("Edge Mask", cv::WND_PROP_VISIBLE) >= 1; }
catch (const cv::Exception&) { return false; }
}
};
// ─────────────────────────────────────────────────────────────────────────────
int main() {
using namespace kpn;
// Nodes — all labelled for web debug UI
auto src = make_node<capture, "capture">(8);
auto gray_node = make_node<to_gray, "to_gray">(8);
auto edge_node = make_node<edges_fn, "edges" >(8);
auto quant = make_node<quantise, "quant" >(8);
auto comp = make_node<composite, "comp" >(8);
// DisplayNodes live on the main thread — registered as sinks
DisplayComposite disp_comp;
DisplayEdges disp_edges;
// make_network() detects that edge_node.output<0>() feeds two consumers
// (comp and disp_edges) and inserts FanoutNode<cv::Mat, 2> automatically.
auto net = make_network(
edge(src.output<0>(), quant.input<0>()), // colour → quant
edge(src.output<1>(), gray_node.input<0>()), // grey → to_gray
edge(gray_node.output<0>(), edge_node.input<0>()), // gray → edges
edge(edge_node.output<0>(), comp.input<0>()), // edges → comp (fan-out src)
edge(edge_node.output<0>(), disp_edges.input<0>()), // edges → display_edges (auto fanout)
edge(quant.output<0>(), comp.input<1>()), // quantised → comp
edge(comp.output<0>(), disp_comp.input<0>()) // result → display_composite
);
std::cout << "Cell-shading pipeline (static) running. Press 'q' to stop.\n";
net.start();
// Main thread drives both display nodes — step() returns false when
// operator() returns false (q pressed or window closed).
while (disp_comp.step() && disp_edges.step())
cv::waitKey(8);
net.stop();
return 0;
}
+288
View File
@@ -0,0 +1,288 @@
// Example 13 — Debug Cell-Shading Pipeline with Tiled Debug Canvas
//
// An improved cell-shading pipeline where every processing node performs
// exactly one OpenCV operation. A variadic DebugCanvas<N> node tiles N
// cv::Mat inputs into a single debug window, making each pipeline stage
// visible side-by-side at runtime.
//
// Improvements over example 12:
// - Bilateral filter before quantisation (edge-preserving smoothing):
// flattens colour regions without softening object edges
// - Dilated edge mask for bolder black outlines
// - 6-level quantisation for richer tonal detail
//
// Topology (auto-fanouts inserted by make_network):
//
// [capture]──┬──> [bilateral]──> [quant]──┬──> [comp]──> [debug:0 result]
// │ └──> [debug:1 quantised]
// ├──> [to_gray]──> [blur]──> [canny]──┬──> [dilate]──> [comp]
// │ └──> [debug:2 edges]
// └──> [debug:3 original]
//
// make_network detects the 3-way fan from [capture], the 2-way fan from
// [quant], and the 2-way fan from [canny], inserting FanoutNode instances
// automatically.
#include <kpn/kpn.hpp>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/videoio.hpp>
#include <algorithm>
#include <chrono>
#include <cmath>
#include <iostream>
#include <string>
#include <thread>
#include <type_traits>
#include <vector>
// ── Synthetic source for environments without a webcam ───────────────────────
static cv::Mat make_gradient(int W, int H) {
cv::Mat xr(H, W, CV_8UC1), yg(H, W, CV_8UC1), b(H, W, CV_8UC1, cv::Scalar(128));
for (int x = 0; x < W; ++x) xr.col(x).setTo(x * 255 / W);
for (int y = 0; y < H; ++y) yg.row(y).setTo(y * 255 / H);
cv::Mat channels[3] = {b, yg, xr};
cv::Mat grad;
cv::merge(channels, 3, grad);
return grad;
}
// ── Pipeline nodes — one cv:: call per function ───────────────────────────────
static cv::Mat capture() {
constexpr int W = 640, H = 480;
static cv::VideoCapture cap;
static bool opened = false;
if (!opened) {
opened = true;
cap.open(0, cv::CAP_V4L2);
if (cap.isOpened()) {
cap.set(cv::CAP_PROP_FRAME_WIDTH, W);
cap.set(cv::CAP_PROP_FRAME_HEIGHT, H);
} else {
std::cerr << "[capture] no webcam — using synthetic animated pattern\n";
}
}
cv::Mat frame;
if (cap.isOpened()) {
auto t0 = std::chrono::steady_clock::now();
cap >> frame;
auto elapsed = std::chrono::steady_clock::now() - t0;
if (elapsed < std::chrono::milliseconds(20))
std::this_thread::sleep_for(std::chrono::milliseconds(33) - elapsed);
if (frame.empty()) frame = cv::Mat::zeros(H, W, CV_8UC3);
} else {
static int tick = 0;
static cv::Mat grad = make_gradient(W, H);
++tick;
frame = grad.clone();
int r = 150 + (tick % 80) * 4;
cv::circle(frame, {W/2, H/2}, r, {255, 200, 0}, -1);
cv::circle(frame, {W/2, H/2}, r / 2, { 0, 128, 255}, -1);
cv::circle(frame, {W*2/5, H*2/5}, r / 3, {200, 0, 200}, -1);
std::this_thread::sleep_for(std::chrono::milliseconds(33));
}
return frame.clone();
}
// Edge-preserving smooth: flattens colour within regions while keeping sharp
// boundaries — much better than Gaussian blur as a pre-quantisation step.
static cv::Mat bilateral_filter(cv::Mat bgr) {
cv::Mat out;
cv::bilateralFilter(bgr, out, 5, 75, 75);
return out;
}
// Snap each channel to N discrete tonal levels.
static cv::Mat quantise(cv::Mat bgr) {
constexpr int levels = 6;
constexpr double step = 256.0 / levels;
static const cv::Mat lut = []() {
cv::Mat l(1, 256, CV_8UC1);
for (int i = 0; i < 256; ++i)
l.at<uchar>(i) = cv::saturate_cast<uchar>(
std::floor(i / step) * step + step / 2.0);
return l;
}();
cv::Mat out;
cv::LUT(bgr, lut, out);
return out;
}
// BGR → greyscale; the edge path works on the original (not bilateral-filtered)
// frame so that fine edge detail is preserved.
static cv::Mat to_gray(cv::Mat bgr) {
cv::Mat gray;
cv::cvtColor(bgr, gray, cv::COLOR_BGR2GRAY);
return gray;
}
// Suppress high-frequency noise before the Canny detector.
static cv::Mat gaussian_blur(cv::Mat gray) {
cv::Mat out;
cv::GaussianBlur(gray, out, {5, 5}, 0);
return out;
}
// Detect strong edges; returns a binary mask (CV_8UC1).
static cv::Mat canny_edges(cv::Mat blurred) {
cv::Mat out;
cv::Canny(blurred, out, 50, 150);
return out;
}
// Widen the edge mask for bolder cartoon outlines.
static cv::Mat dilate_edges(cv::Mat edges) {
static const cv::Mat kernel = cv::getStructuringElement(cv::MORPH_RECT, {3, 3});
cv::Mat out;
cv::dilate(edges, out, kernel);
return out;
}
// Burn black outlines into the quantised colour image.
static cv::Mat composite(cv::Mat quantised, cv::Mat thick_edges) {
cv::Mat result = quantised.clone();
result.setTo(cv::Scalar(0, 0, 0), thick_edges);
return result;
}
// ── DebugCanvas<N> ────────────────────────────────────────────────────────────
//
// Variadic MainThreadNode that accepts N cv::Mat inputs (any mix of BGR and
// greyscale) and tiles them into one debug window arranged as a
// ceil(sqrt(N)) × ceil(N/cols) grid.
//
// Template trick: MatArg<I> aliases cv::Mat for all I, so the pack expansion
// MatArg<0>, MatArg<1>, ..., MatArg<N-1>
// produces exactly N cv::Mat arguments — enough to drive the MainThreadNode
// base without manually spelling out the type N times.
template<std::size_t>
using MatArg = cv::Mat;
template<std::size_t N>
class DebugCanvas;
template<std::size_t N, typename Seq = std::make_index_sequence<N>>
struct DebugCanvasBase;
template<std::size_t N, std::size_t... Is>
struct DebugCanvasBase<N, std::index_sequence<Is...>> {
using type = kpn::MainThreadNode<DebugCanvas<N>, kpn::in<>, MatArg<Is>...>;
};
template<std::size_t N>
class DebugCanvas : public DebugCanvasBase<N>::type {
using Base = typename DebugCanvasBase<N>::type;
public:
static constexpr std::string_view label() { return "debug_canvas"; }
static constexpr std::size_t unique_tag = 0;
explicit DebugCanvas(std::vector<std::string> slot_labels = {},
std::size_t fifo_cap = 4)
: Base(fifo_cap), labels_(std::move(slot_labels))
{
cv::namedWindow("Debug Canvas", cv::WINDOW_NORMAL);
cv::resizeWindow("Debug Canvas", cols() * CW, rows() * CH);
}
~DebugCanvas() { cv::destroyWindow("Debug Canvas"); }
// Called by MainThreadNode::step() with exactly N cv::Mat arguments.
template<typename... Ms>
bool operator()(Ms&&... mats) {
static_assert(sizeof...(Ms) == N, "DebugCanvas: wrong number of inputs");
std::vector<cv::Mat> imgs;
imgs.reserve(N);
(imgs.push_back(to_bgr(std::forward<Ms>(mats))), ...);
cv::imshow("Debug Canvas", tile(imgs));
int key = cv::waitKey(1);
if (key == 'q' || key == 27) return false;
try { return cv::getWindowProperty("Debug Canvas", cv::WND_PROP_VISIBLE) >= 1; }
catch (const cv::Exception&) { return false; }
}
private:
static constexpr int CW = 640, CH = 480;
static int cols() { return std::max(1, (int)std::ceil(std::sqrt((double)N))); }
static int rows() { return ((int)N + cols() - 1) / cols(); }
std::vector<std::string> labels_;
static cv::Mat to_bgr(const cv::Mat& m) {
if (m.channels() == 1) {
cv::Mat bgr;
cv::cvtColor(m, bgr, cv::COLOR_GRAY2BGR);
return bgr;
}
return m;
}
cv::Mat tile(const std::vector<cv::Mat>& imgs) const {
const int c = cols(), r = rows();
cv::Mat canvas(r * CH, c * CW, CV_8UC3, cv::Scalar(30, 30, 30));
for (int i = 0; i < (int)imgs.size(); ++i) {
if (imgs[i].empty()) continue;
cv::Mat cell = canvas(cv::Rect((i % c) * CW, (i / c) * CH, CW, CH));
cv::Mat resized;
cv::resize(imgs[i], resized, {CW, CH});
resized.copyTo(cell);
if (i < (int)labels_.size() && !labels_[i].empty())
cv::putText(cell, labels_[i], {8, 36},
cv::FONT_HERSHEY_SIMPLEX, 1.0, {0, 255, 255}, 2,
cv::LINE_AA);
}
return canvas;
}
};
// ── main ──────────────────────────────────────────────────────────────────────
int main() {
using namespace kpn;
auto src = make_node<capture, "capture" >(4);
auto bilateral= make_node<bilateral_filter, "bilateral" >(4);
auto quant = make_node<quantise, "quant" >(4);
auto gray = make_node<to_gray, "to_gray" >(4);
auto blur = make_node<gaussian_blur, "blur" >(4);
auto canny = make_node<canny_edges, "canny" >(4);
auto dilate = make_node<dilate_edges, "dilate" >(4);
auto comp = make_node<composite, "comp" >(4);
DebugCanvas<4> debug({"result", "quantised", "edges", "original"});
// make_network auto-inserts FanoutNode instances wherever a source port
// feeds more than one consumer:
// capture → 3-way fanout (bilateral, to_gray, debug[3])
// quant → 2-way fanout (comp, debug[1])
// canny → 2-way fanout (dilate, debug[2])
auto net = make_network(
edge(src.output<0>(), bilateral.input<0>()), // frame → bilateral
edge(src.output<0>(), gray.input<0>()), // frame → to_gray
edge(src.output<0>(), debug.input<3>()), // frame → debug[3] original
edge(bilateral.output<0>(), quant.input<0>()), // smooth → quant
edge(quant.output<0>(), comp.input<0>()), // quant → comp
edge(quant.output<0>(), debug.input<1>()), // quant → debug[1] quantised
edge(gray.output<0>(), blur.input<0>()), // gray → blur
edge(blur.output<0>(), canny.input<0>()), // blurred→ canny
edge(canny.output<0>(), dilate.input<0>()), // edges → dilate
edge(canny.output<0>(), debug.input<2>()), // edges → debug[2] edges
edge(dilate.output<0>(), comp.input<1>()), // thick → comp
edge(comp.output<0>(), debug.input<0>()) // result → debug[0] result
);
std::cout << "Debug cell-shading pipeline running — press 'q' to stop.\n";
std::cout << "Canvas: [0] result [1] quantised [2] edges [3] original\n";
net.start();
while (debug.step())
cv::waitKey(8);
net.stop();
return 0;
}
+159
View File
@@ -0,0 +1,159 @@
// Example 14 — DebugHub with Shared Resource Token
//
// Two independent KPN networks compete for one shared inference resource
// (simulating a small GPU or single-session ONNX runtime). The DebugHub
// serves a single web UI at http://localhost:9090 with:
//
// [All Networks] — resource utilisation cards + cross-network node table
// [detect] — force-directed graph for the detection pipeline
// [classify] — force-directed graph for the classification pipeline
//
// Topology:
//
// detect pipeline:
// [source_detect] ──> [run_detect] ──> [sink_detect]
//
// classify pipeline:
// [source_classify] ──> [run_classify] ──> [sink_classify]
//
// Both [run_detect] and [run_classify] call gpu.acquire() before touching the
// simulated device. The priority-based token awards the next slot to the
// waiter that is more likely to make useful progress (higher priority score).
//
// Build: cmake -DKPN_WEB_DEBUG=ON .. && cmake --build .
// Run: ./14_debug_hub
// UI: http://localhost:9090
#ifdef KPN_WEB_DEBUG
#include <kpn/kpn.hpp>
#include <atomic>
#include <chrono>
#include <iostream>
#include <thread>
using namespace kpn;
using namespace std::chrono_literals;
// ── Simulated inference device ────────────────────────────────────────────────
//
// Represents any exclusive, serialised accelerator: GPU session, ONNX runtime,
// hardware encoder, etc. Only one caller can hold it at a time.
struct GPU {
// Detection model: fast, 8 ms per frame.
int detect(int frame_id) {
std::this_thread::sleep_for(8ms);
return frame_id * 2; // synthetic "score"
}
// Classification model: heavier, 14 ms per frame.
int classify(int frame_id) {
std::this_thread::sleep_for(14ms);
return frame_id % 10; // synthetic "label"
}
};
// Global pointer so free-function nodes can reach the resource.
// In production code, capture by reference inside an ObjectNode functor instead.
static SharedResource<GPU>* g_gpu = nullptr;
// ── Detection pipeline ────────────────────────────────────────────────────────
static int source_detect() {
static std::atomic<int> id{0};
std::this_thread::sleep_for(25ms); // ~40 fps source rate
return id.fetch_add(1, std::memory_order_relaxed);
}
static int run_detect(int frame_id) {
// Higher priority: detection is latency-critical.
auto guard = g_gpu->acquire([] { return 0.7f; });
return guard->detect(frame_id);
}
static std::atomic<uint64_t> detect_out{0};
static void sink_detect(int) {
detect_out.fetch_add(1, std::memory_order_relaxed);
}
// ── Classification pipeline ───────────────────────────────────────────────────
static int source_classify() {
static std::atomic<int> id{0};
std::this_thread::sleep_for(40ms); // ~25 fps source rate
return id.fetch_add(1, std::memory_order_relaxed);
}
static int run_classify(int frame_id) {
// Lower priority: classification is best-effort.
auto guard = g_gpu->acquire([] { return 0.3f; });
return guard->classify(frame_id);
}
static std::atomic<uint64_t> classify_out{0};
static void sink_classify(int) {
classify_out.fetch_add(1, std::memory_order_relaxed);
}
// ── main ──────────────────────────────────────────────────────────────────────
int main() {
SharedResource<GPU> gpu;
g_gpu = &gpu;
// ── Detection network ─────────────────────────────────────────────────────
auto src_det = make_node<source_detect, "source_detect">(4);
auto inf_det = make_node<run_detect, "run_detect" >(4);
auto snk_det = make_node<sink_detect, "sink_detect" >(4);
auto net_detect = make_network(
edge(src_det.output<0>(), inf_det.input<0>()),
edge(inf_det.output<0>(), snk_det.input<0>())
);
// ── Classification network ────────────────────────────────────────────────
auto src_cls = make_node<source_classify, "source_classify">(4);
auto inf_cls = make_node<run_classify, "run_classify" >(4);
auto snk_cls = make_node<sink_classify, "sink_classify" >(4);
auto net_classify = make_network(
edge(src_cls.output<0>(), inf_cls.input<0>()),
edge(inf_cls.output<0>(), snk_cls.input<0>())
);
// ── Hub — one debug server for both networks + the shared resource ─────────
DebugHub hub(9090);
hub.register_network("detect", net_detect);
hub.register_network("classify", net_classify);
hub.register_resource("gpu", &gpu);
net_detect.start();
net_classify.start();
hub.start();
std::cout << "Running — open http://localhost:9090\n"
<< "Tabs: [All Networks] [detect] [classify]\n"
<< "Press Enter to stop.\n";
std::cin.get();
net_detect.stop();
net_classify.stop();
std::cout << "\nResults:\n"
<< " detect: " << detect_out.load() << " frames\n"
<< " classify: " << classify_out.load() << " frames\n";
return 0;
}
#else // no KPN_WEB_DEBUG
#include <iostream>
int main() {
std::cerr << "This example requires KPN_WEB_DEBUG.\n"
<< "Rebuild with: cmake -DKPN_WEB_DEBUG=ON ..\n";
return 1;
}
#endif
+75
View File
@@ -0,0 +1,75 @@
// Example 15 — Per-node Error Handler
//
// Demonstrates set_error_handler() for deciding whether a network can
// continue when a node throws an exception.
//
// The "validator" node rejects even numbers by throwing std::runtime_error.
// Its error handler logs the failure and returns true (skip & continue),
// so odd numbers still flow through to the sink.
//
// Compare: a second handler (commented below) returns false instead,
// which stops the node and gracefully shuts the downstream side down.
//
// Pipeline: [source] --int--> [validator] --int--> [sink]
#include <kpn/kpn.hpp>
#include <chrono>
#include <iostream>
#include <thread>
static int counter = 0;
static int source() {
std::this_thread::sleep_for(std::chrono::milliseconds(20));
return ++counter;
}
static int validate(int x) {
if (x % 2 == 0)
throw std::runtime_error("even number rejected: " + std::to_string(x));
return x;
}
static int received = 0;
static void sink(int x) {
std::cout << " processed: " << x << '\n';
++received;
}
int main() {
using namespace kpn;
auto src = make_node<source> ();
auto proc = make_node<validate>();
auto snk = make_node<sink> ();
// --8<-- [start:error_handler]
// Return true → skip this invocation, keep the node running.
// Return false → stop the node (downstream drains then also stops).
proc.set_error_handler([](std::string_view name, std::exception_ptr ep) {
try { std::rethrow_exception(ep); }
catch (const std::exception& e) {
std::cerr << "[" << name << "] skipping item — " << e.what() << '\n';
}
return true;
});
// --8<-- [end:error_handler]
Network net;
net.add("source", src)
.add("validator", proc)
.add("sink", snk)
.connect("source", src.output<0>(), "validator", proc.input<0>())
.connect("validator", proc.output<0>(), "sink", snk.input<0>())
.build();
std::cout << "source emits 1..N; validator rejects even numbers.\n"
<< "Error messages on stderr, accepted items on stdout.\n\n";
net.start();
std::this_thread::sleep_for(std::chrono::milliseconds(300));
net.stop();
std::cout << "\nItems accepted by sink: " << received << '\n';
}
+83
View File
@@ -0,0 +1,83 @@
// Example 16 — Event Callbacks: overflow and node-stopped signals
//
// Two complementary observation mechanisms:
//
// 1. Per-node overflow callback set_overflow_callback()
// Fired (with a timestamp) when a node's output channel is full and an
// item is dropped. Useful for targeted monitoring of a specific node.
//
// 2. Network-level event handler net.set_event_handler()
// Aggregate callback covering every node: receives the node name, a
// NodeEvent (Overflow or Closed), and a timestamp. Register once and
// observe the whole network.
//
// Pipeline: [fast_source] --int--> [slow_sink]
//
// fast_source produces at ~500 items/s; slow_sink consumes at ~20 items/s.
// The channel capacity is 3, so overflows appear within milliseconds.
#include <kpn/kpn.hpp>
#include <atomic>
#include <chrono>
#include <iostream>
#include <thread>
using namespace kpn;
using namespace std::chrono;
// ── Node functions ────────────────────────────────────────────────────────────
// --8<-- [start:node_fns]
static std::atomic<int> g_seq{0};
static int fast_source() {
std::this_thread::sleep_for(milliseconds(2)); // ~500/s
return g_seq.fetch_add(1);
}
static void slow_sink(int x) {
std::this_thread::sleep_for(milliseconds(50)); // ~20/s
std::cout << " consumed: " << x << '\n';
}
// --8<-- [end:node_fns]
// ── main ──────────────────────────────────────────────────────────────────────
int main() {
auto src = make_node<fast_source>(/*capacity=*/3);
auto snk = make_node<slow_sink> (/*capacity=*/3);
// --8<-- [start:per_node_callback]
// Per-node overflow callback — no node name needed, known at registration.
std::atomic<int> overflow_count{0};
src.set_overflow_callback([&](steady_clock::time_point ts) {
auto ms = duration_cast<milliseconds>(ts.time_since_epoch()).count();
std::cerr << "[overflow] fast_source at t=" << ms << "ms\n";
overflow_count.fetch_add(1);
});
// --8<-- [end:per_node_callback]
Network net;
// --8<-- [start:network_event_handler]
// Network-level aggregate handler — covers every node, includes node name.
net.set_event_handler([](std::string_view name, NodeEvent ev,
steady_clock::time_point ts) {
auto ms = duration_cast<milliseconds>(ts.time_since_epoch()).count();
std::string_view kind = (ev == NodeEvent::Overflow) ? "overflow" : "closed";
std::cerr << "[net:" << kind << "] node=" << name << " t=" << ms << "ms\n";
});
// --8<-- [end:network_event_handler]
net.add("source", src)
.add("sink", snk)
.connect("source", src.output<0>(), "sink", snk.input<0>())
.build()
.start();
std::this_thread::sleep_for(milliseconds(300));
net.stop();
std::cout << "\nTotal overflows observed by per-node callback: "
<< overflow_count.load() << '\n';
}
+83
View File
@@ -0,0 +1,83 @@
cmake_minimum_required(VERSION 3.21)
# Build an example and register it as a CTest smoke test.
# Examples that are self-terminating (fixed sleep net.stop()) pass when
# they exit 0 within TIMEOUT seconds. OpenCV/UI examples are excluded.
function(kpn_example name)
add_executable(${name} ${name}/main.cpp)
target_link_libraries(${name} PRIVATE kpn)
add_test(NAME example_${name} COMMAND ${name})
set_tests_properties(example_${name} PROPERTIES
TIMEOUT 15
LABELS examples
)
endfunction()
# Register a Python example script as a CTest smoke test. Runs the script with
# PYTHONPATH pointing at the freshly-built kpn_python module, so it does not
# depend on the caller's working directory or a hard-coded "build/python" path.
function(kpn_python_example name)
if(NOT KPN_BUILD_PYTHON)
return()
endif()
add_test(
NAME example_${name}
COMMAND ${CMAKE_COMMAND} -E env
"PYTHONPATH=$<TARGET_FILE_DIR:kpn_python>"
${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/${name}/example.py
)
set_tests_properties(example_${name} PROPERTIES
TIMEOUT 15
LABELS examples
)
endfunction()
kpn_example(01_hello_pipeline)
kpn_example(02_named_ports)
kpn_example(03_multi_output)
kpn_example(04_storage_policy)
kpn_example(05_error_handling)
kpn_example(06_watchdog)
set_tests_properties(example_06_watchdog PROPERTIES TIMEOUT 40)
kpn_example(10_static_hello_pipeline)
kpn_example(11_static_fanout)
kpn_example(15_node_error_handler)
kpn_example(16_event_callbacks)
if(KPN_WEB_DEBUG)
kpn_target_enable_web_debug(06_watchdog)
add_executable(14_debug_hub 14_debug_hub/main.cpp)
target_link_libraries(14_debug_hub PRIVATE kpn)
kpn_target_enable_web_debug(14_debug_hub)
endif()
# 07 and 08 are Python scripts no compiled target, but run as smoke tests.
kpn_python_example(07_python_network)
kpn_python_example(08_python_subport)
# 09 requires OpenCV only build if found
find_package(OpenCV QUIET COMPONENTS core imgproc highgui videoio)
if(OpenCV_FOUND)
# Hybrid Python example: kpn_opencv module (requires both OpenCV and nanobind)
if(KPN_BUILD_PYTHON)
nanobind_add_module(kpn_opencv 09_opencv_cellshade/kpn_opencv.cpp)
target_link_libraries(kpn_opencv PRIVATE kpn ${OpenCV_LIBS})
target_compile_definitions(kpn_opencv PRIVATE KPN_BUILD_PYTHON)
message(STATUS "KPN++ kpn_opencv Python module: building (OpenCV ${OpenCV_VERSION})")
endif()
add_executable(09_opencv_cellshade 09_opencv_cellshade/main.cpp)
target_link_libraries(09_opencv_cellshade PRIVATE kpn ${OpenCV_LIBS})
add_executable(12_static_cellshade 12_static_cellshade/main.cpp)
target_link_libraries(12_static_cellshade PRIVATE kpn ${OpenCV_LIBS})
add_executable(13_debug_cellshade 13_debug_cellshade/main.cpp)
target_link_libraries(13_debug_cellshade PRIVATE kpn ${OpenCV_LIBS})
if(KPN_WEB_DEBUG)
kpn_target_enable_web_debug(09_opencv_cellshade)
kpn_target_enable_web_debug(13_debug_cellshade)
endif()
message(STATUS "KPN++ example 09_opencv_cellshade: OpenCV ${OpenCV_VERSION} found — building")
else()
message(STATUS "KPN++ example 09_opencv_cellshade: OpenCV not found — skipping")
endif()
-1039
View File
File diff suppressed because it is too large Load Diff
-1013
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+301
View File
@@ -0,0 +1,301 @@
#pragma once
#include "channel.hpp"
#include "diagnostics.hpp"
#include "inode.hpp"
#include "port.hpp"
#include "traits.hpp"
#include <array>
#include <atomic>
#include <functional>
#include <memory>
#include <thread>
namespace kpn {
// ── RouterNode ────────────────────────────────────────────────────────────────
//
// Reads one item and pushes it to exactly one of N output channels, chosen by
// selector(item). If selector returns >= N the item is silently dropped.
//
// Usage:
// auto router = make_router<Image, 3>(
// [](const Image& img) -> std::size_t { return img.stream_id % 3; });
// net.connect("src", src.output<0>(), "router", router.input<0>())
// .connect("router", router.output<0>(), "nodeA", nodeA.input<0>())
// .connect("router", router.output<1>(), "nodeB", nodeB.input<0>())
// .connect("router", router.output<2>(), "nodeC", nodeC.input<0>());
template<typename T, std::size_t N, std::size_t Id = 0>
class RouterNode : public INode {
public:
using Selector = std::function<std::size_t(const T&)>;
using args_tuple = std::tuple<T>;
using return_tuple = repeat_tuple_t<T, N>;
using return_raw = return_tuple;
static constexpr std::size_t input_count = 1;
static constexpr std::size_t output_count = N;
static constexpr std::size_t unique_tag = Id;
static constexpr bool is_router_node = true;
explicit RouterNode(Selector sel, std::size_t fifo_capacity = 5)
: selector_(std::move(sel))
, fifo_capacity_(fifo_capacity)
{
input_ch_ = std::make_shared<Channel<T>>(fifo_capacity);
}
~RouterNode() override { stop(); }
// ── INode ─────────────────────────────────────────────────────────────────
void start() override {
input_ch_->enable();
stop_flag_.store(false, std::memory_order_relaxed);
thread_ = std::jthread([this](std::stop_token) { run_loop(); });
}
void stop() override {
stop_flag_.store(true, std::memory_order_relaxed);
input_ch_->disable();
if (thread_.joinable()) thread_.request_stop(), thread_.join();
}
bool running() const override {
return thread_.joinable() && !stop_flag_.load(std::memory_order_relaxed);
}
void set_name(std::string name) override { name_ = std::move(name); }
const NodeStats& stats() const override { return stats_; }
NodeSnapshot node_snapshot(const std::string& name, double elapsed_s) const override {
uint64_t frames = stats_.frames_processed.load(std::memory_order_relaxed);
double exec_ms = stats_.ema_exec_us.load(std::memory_order_relaxed) / 1000.0;
double blocked_ms = stats_.total_blocked_us.load(std::memory_order_relaxed) / 1000.0;
double total_ms = exec_ms + blocked_ms;
return {name, frames, exec_ms,
stats_.max_exec_us.load(std::memory_order_relaxed) / 1000.0,
blocked_ms,
elapsed_s > 0 ? frames / elapsed_s : 0.0,
stats_.total_cpu_us.load(std::memory_order_relaxed) / 1000.0,
total_ms > 0 ? 100.0 * exec_ms / total_ms : 0.0};
}
// ── Port access ───────────────────────────────────────────────────────────
template<std::size_t I = 0>
InputPort<RouterNode, I> input() {
static_assert(I == 0, "RouterNode has exactly one input");
return {*this};
}
template<std::size_t I>
OutputPort<RouterNode, I> output() {
static_assert(I < N, "RouterNode output index out of range");
return {*this};
}
// ── Internal channel accessors (called by Network::connect) ───────────────
template<std::size_t I>
Channel<T>& input_channel() {
static_assert(I == 0);
return *input_ch_;
}
template<std::size_t I>
void set_input_channel(std::shared_ptr<Channel<T>> ch) {
static_assert(I == 0);
input_ch_ = std::move(ch);
}
template<std::size_t I>
void set_output_channel(Channel<T>* ch) {
static_assert(I < N);
out_channels_[I] = ch;
}
private:
void run_loop() {
while (!stop_flag_.load(std::memory_order_relaxed)) {
try {
auto t0 = clock_t::now();
T val = input_ch_->pop();
auto t1 = clock_t::now();
auto cpu0 = NodeStats::cpu_now();
std::size_t idx = selector_(val);
if (idx < N && out_channels_[idx]) {
try { out_channels_[idx]->push(val); }
catch (const ChannelOverflowError&) {}
}
auto cpu1 = NodeStats::cpu_now();
auto t2 = clock_t::now();
stats_.record_exec(duration_t(t2 - t1), duration_t(t1 - t0), cpu0, cpu1);
} catch (const ChannelClosedError&) {
break;
}
}
}
std::string name_;
std::size_t fifo_capacity_;
Selector selector_;
std::shared_ptr<Channel<T>> input_ch_;
std::array<Channel<T>*, N> out_channels_{};
std::atomic<bool> stop_flag_{false};
std::jthread thread_;
NodeStats stats_;
};
// ── FilterNode ────────────────────────────────────────────────────────────────
//
// Reads one item and pushes it downstream only when pred(item) is true.
// Dropped items are not counted as processed frames.
//
// Usage:
// auto filt = make_filter<Frame>([](const Frame& f) { return f.valid; });
// net.connect("src", src.output<0>(), "filt", filt.input<0>())
// .connect("filt", filt.output<0>(), "dst", dst.input<0>());
template<typename T, std::size_t Id = 0>
class FilterNode : public INode {
public:
using Predicate = std::function<bool(const T&)>;
using args_tuple = std::tuple<T>;
using return_tuple = std::tuple<T>;
using return_raw = return_tuple;
static constexpr std::size_t input_count = 1;
static constexpr std::size_t output_count = 1;
static constexpr std::size_t unique_tag = Id;
static constexpr bool is_filter_node = true;
explicit FilterNode(Predicate pred, std::size_t fifo_capacity = 5)
: pred_(std::move(pred))
, fifo_capacity_(fifo_capacity)
{
input_ch_ = std::make_shared<Channel<T>>(fifo_capacity);
}
~FilterNode() override { stop(); }
// ── INode ─────────────────────────────────────────────────────────────────
void start() override {
input_ch_->enable();
stop_flag_.store(false, std::memory_order_relaxed);
thread_ = std::jthread([this](std::stop_token) { run_loop(); });
}
void stop() override {
stop_flag_.store(true, std::memory_order_relaxed);
input_ch_->disable();
if (thread_.joinable()) thread_.request_stop(), thread_.join();
}
bool running() const override {
return thread_.joinable() && !stop_flag_.load(std::memory_order_relaxed);
}
void set_name(std::string name) override { name_ = std::move(name); }
const NodeStats& stats() const override { return stats_; }
NodeSnapshot node_snapshot(const std::string& name, double elapsed_s) const override {
uint64_t frames = stats_.frames_processed.load(std::memory_order_relaxed);
double exec_ms = stats_.ema_exec_us.load(std::memory_order_relaxed) / 1000.0;
double blocked_ms = stats_.total_blocked_us.load(std::memory_order_relaxed) / 1000.0;
double total_ms = exec_ms + blocked_ms;
return {name, frames, exec_ms,
stats_.max_exec_us.load(std::memory_order_relaxed) / 1000.0,
blocked_ms,
elapsed_s > 0 ? frames / elapsed_s : 0.0,
stats_.total_cpu_us.load(std::memory_order_relaxed) / 1000.0,
total_ms > 0 ? 100.0 * exec_ms / total_ms : 0.0};
}
// ── Port access ───────────────────────────────────────────────────────────
template<std::size_t I = 0>
InputPort<FilterNode, I> input() {
static_assert(I == 0, "FilterNode has exactly one input");
return {*this};
}
template<std::size_t I = 0>
OutputPort<FilterNode, I> output() {
static_assert(I == 0, "FilterNode has exactly one output");
return {*this};
}
// ── Internal channel accessors (called by Network::connect) ───────────────
template<std::size_t I>
Channel<T>& input_channel() {
static_assert(I == 0);
return *input_ch_;
}
template<std::size_t I>
void set_input_channel(std::shared_ptr<Channel<T>> ch) {
static_assert(I == 0);
input_ch_ = std::move(ch);
}
template<std::size_t I>
void set_output_channel(Channel<T>* ch) {
static_assert(I == 0);
out_ch_ = ch;
}
private:
void run_loop() {
while (!stop_flag_.load(std::memory_order_relaxed)) {
try {
auto t0 = clock_t::now();
T val = input_ch_->pop();
auto t1 = clock_t::now();
auto cpu0 = NodeStats::cpu_now();
if (pred_(val) && out_ch_) {
try { out_ch_->push(val); }
catch (const ChannelOverflowError&) {}
auto cpu1 = NodeStats::cpu_now();
auto t2 = clock_t::now();
stats_.record_exec(duration_t(t2 - t1), duration_t(t1 - t0), cpu0, cpu1);
}
} catch (const ChannelClosedError&) {
break;
}
}
}
std::string name_;
std::size_t fifo_capacity_;
Predicate pred_;
std::shared_ptr<Channel<T>> input_ch_;
Channel<T>* out_ch_{nullptr};
std::atomic<bool> stop_flag_{false};
std::jthread thread_;
NodeStats stats_;
};
// ── Factories ─────────────────────────────────────────────────────────────────
template<typename T, std::size_t N>
RouterNode<T, N> make_router(std::function<std::size_t(const T&)> sel,
std::size_t capacity = 5) {
return RouterNode<T, N, 0>(std::move(sel), capacity);
}
template<typename T>
FilterNode<T> make_filter(std::function<bool(const T&)> pred,
std::size_t capacity = 5) {
return FilterNode<T, 0>(std::move(pred), capacity);
}
} // namespace kpn
+443
View File
@@ -0,0 +1,443 @@
#pragma once
#include "diagnostics.hpp"
#include <atomic>
#include <chrono>
#include <cstdint>
#include <functional>
#include <memory>
#include <stdexcept>
#include <string>
#include <thread>
#include <type_traits>
namespace kpn {
// ── Data size trait ───────────────────────────────────────────────────────────
// Returns the number of bytes of logical payload carried by a value.
// Defaults to sizeof(T), which is correct for PODs and fixed-size types.
// Specialize for heap-owning types (e.g. cv::Mat) to get accurate bandwidth:
//
// template<> struct kpn::ChannelDataSize<cv::Mat> {
// static std::size_t bytes(const cv::Mat& m) { return m.total() * m.elemSize(); }
// };
template<typename T>
struct ChannelDataSize {
static std::size_t bytes(const T&) { return sizeof(T); }
};
// ── Storage policy ────────────────────────────────────────────────────────────
template<typename T>
struct channel_storage_policy {
static constexpr bool by_value =
std::is_trivially_copyable_v<T> && sizeof(T) <= 8;
};
template<typename T>
using channel_storage_t = std::conditional_t<
channel_storage_policy<T>::by_value,
T,
std::shared_ptr<const T>
>;
// ── Exceptions ────────────────────────────────────────────────────────────────
class ChannelOverflowError : public std::runtime_error {
public:
explicit ChannelOverflowError(std::size_t capacity)
: std::runtime_error("channel overflow: capacity " + std::to_string(capacity) +
" exceeded") {}
ChannelOverflowError(std::size_t capacity, std::string context)
: std::runtime_error(std::move(context) + ": capacity " + std::to_string(capacity) +
" exceeded") {}
};
class ChannelClosedError : public std::runtime_error {
public:
ChannelClosedError() : std::runtime_error("channel closed") {}
};
// ── CPU pause hint ────────────────────────────────────────────────────────────
// Signals the CPU that this is a spin-wait loop, improving HT sibling throughput
// and preventing branch-predictor thrash on x86. Falls back to a compiler barrier.
[[maybe_unused]] static void spin_hint() noexcept {
#if defined(__x86_64__) || defined(__i386__)
__asm__ volatile("pause" ::: "memory");
#elif defined(__aarch64__) || defined(__arm__)
__asm__ volatile("yield" ::: "memory");
#else
std::atomic_signal_fence(std::memory_order_seq_cst);
#endif
}
// ── Channel ───────────────────────────────────────────────────────────────────
// SPSC ring buffer with atomic wait/notify and configurable spin-before-sleep.
//
// `spin_count` (constructor arg, default 200): number of pause-hint iterations
// before falling back to atomic::wait (futex). At ~20 ns/pause on x86 this is
// ~4 µs. Set to 0 to disable spinning (useful for power-constrained or
// predominantly-idle pipelines).
//
// Memory ordering contract (SPSC):
// push(): tail_.store(release) pairs with pop()'s tail_.load(acquire)
// head_.load(acquire) pairs with pop()'s head_.store(release)
// pop(): head_.store(release) pairs with push()'s head_.load(acquire)
// tail_.load(acquire) pairs with push()'s tail_.store(release)
template<typename T>
class Channel {
public:
using storage_type = channel_storage_t<T>;
explicit Channel(std::size_t capacity = 5, std::size_t spin_count = 200)
: capacity_(capacity), spin_count_(spin_count)
{
std::size_t rs = 1;
while (rs <= capacity) rs <<= 1; // smallest power-of-2 > capacity
ring_mask_ = rs - 1;
buf_ = std::make_unique<storage_type[]>(rs);
}
Channel(const Channel&) = delete;
Channel& operator=(const Channel&) = delete;
// Push a value.
// - If channel is disabled (accepting_ == false): silently drop.
// - If channel is full (fill >= capacity_): throw ChannelOverflowError.
void push(T value) {
if (!accepting_.load(std::memory_order_relaxed)) {
stats_.record_drop();
return;
}
const std::size_t data_bytes = ChannelDataSize<T>::bytes(value);
const std::size_t t = tail_.load(std::memory_order_relaxed);
const std::size_t h = head_.load(std::memory_order_acquire);
if (!accepting_.load(std::memory_order_acquire)) {
stats_.record_drop();
return;
}
if (t - h >= capacity_) {
stats_.record_overflow();
throw ChannelOverflowError(capacity_);
}
const bool was_empty = (t == h);
buf_[t & ring_mask_] = make_storage(std::move(value));
tail_.store(t + 1, std::memory_order_release);
stats_.record_push(t - h + 1, data_bytes);
wake_.fetch_add(1, std::memory_order_release);
wake_.notify_one();
if (was_empty && push_callback_)
push_callback_();
}
/// Called when a pop frees a slot in a previously-full ring.
///
/// The mirror of `set_push_callback`, and it exists for the same reason:
/// a producer must be able to *park* rather than spin. Without it the only
/// lossless option is `push_blocking`, which sleeps inside the caller's
/// thread — and when that thread is a scheduler worker, parking it starves
/// every node pinned to it (see the hold-and-wait note on push_sentinel).
void set_space_callback(std::function<void()> cb) { space_callback_ = std::move(cb); }
/// True when a push would currently succeed. Used to close the lost-wakeup
/// race: a producer that parks must re-check after clearing its queued flag,
/// because a space_callback fired in between would otherwise be swallowed.
bool has_space() const {
return tail_.load(std::memory_order_relaxed) -
head_.load(std::memory_order_acquire) < capacity_;
}
/// Non-blocking, lossless push. Returns false when the ring is full, having
/// changed nothing — the caller keeps the value and retries when woken.
bool try_push(T& value) {
if (!accepting_.load(std::memory_order_acquire)) { stats_.record_drop(); return true; }
const std::size_t t = tail_.load(std::memory_order_relaxed);
const std::size_t h = head_.load(std::memory_order_acquire);
if (t - h >= capacity_) return false;
const std::size_t data_bytes = ChannelDataSize<T>::bytes(value);
const bool was_empty = (t == h);
buf_[t & ring_mask_] = make_storage(std::move(value));
tail_.store(t + 1, std::memory_order_release);
stats_.record_push(t - h + 1, data_bytes);
wake_.fetch_add(1, std::memory_order_release);
wake_.notify_one();
if (was_empty && push_callback_) push_callback_();
return true;
}
// Lossless push with BACKPRESSURE: if the ring is full, wait for the consumer to
// drain instead of dropping (the throwing push()) — the producer just runs slower.
// Use when every value must be delivered (e.g. replaying a dump for scoring, where
// a dropped frame silently corrupts the result). SPSC: only the sole producer may
// call it. Returns false if the channel was disabled while waiting.
bool push_blocking(T value) {
for (;;) {
if (!accepting_.load(std::memory_order_acquire)) {
stats_.record_drop();
return false;
}
const std::size_t t = tail_.load(std::memory_order_relaxed);
const std::size_t h = head_.load(std::memory_order_acquire);
if (t - h < capacity_) { // space available → normal push
const std::size_t data_bytes = ChannelDataSize<T>::bytes(value);
const bool was_empty = (t == h);
buf_[t & ring_mask_] = make_storage(std::move(value));
tail_.store(t + 1, std::memory_order_release);
stats_.record_push(t - h + 1, data_bytes);
wake_.fetch_add(1, std::memory_order_release);
wake_.notify_one();
if (was_empty && push_callback_) push_callback_();
return true;
}
// full: yield briefly and retry (consumer will drain)
std::this_thread::sleep_for(std::chrono::microseconds(50));
}
}
// Lossless, non-blocking delivery for a must-deliver control token (EOF).
//
// A sentinel is stored out-of-band — in a dedicated slot that does NOT
// consume ring capacity — so this can never overflow and never blocks the
// caller. That distinction is essential: each KPN node has a single worker
// thread, so a *blocking* push would park that thread and stop it draining
// its own input, cascading into a hold-and-wait deadlock under backpressure.
// Setting a flag and returning keeps the worker free to keep popping.
//
// The consumer's pop() drains the ring first, then delivers this sentinel,
// preserving ordering (EOF arrives after all data pushed before it).
//
// Only the sole producer may call it (SPSC contract, same as push()).
// Returns false if the channel is already disabled (token discarded —
// teardown is in progress, so the sentinel is moot).
bool push_sentinel(T value) {
if (!accepting_.load(std::memory_order_acquire)) {
stats_.record_drop();
return false;
}
eof_value_ = make_storage(std::move(value));
has_eof_.store(true, std::memory_order_release);
// Wake a consumer blocked in pop(): the sentinel is now deliverable even
// though the ring may be empty.
wake_.fetch_add(1, std::memory_order_release);
wake_.notify_one();
if (push_callback_) push_callback_();
return true;
}
// Blocking pop. Returns when an item is available.
// Throws ChannelClosedError if the channel is disabled (regardless of fill).
T pop() {
for (;;) {
// Snapshot wake_ BEFORE reading tail_ to prevent lost wakeups.
const uint32_t w = wake_.load(std::memory_order_relaxed);
const std::size_t h = head_.load(std::memory_order_relaxed);
std::size_t t = tail_.load(std::memory_order_acquire);
// If empty, spin before sleeping: avoids the futex when the next item
// arrives within the spin window (~4 µs at default spin_count=200 on x86).
if (h == t) {
// Ring drained — deliver any pending out-of-band sentinel (EOF)
// now, so it always arrives after the data pushed before it.
//
// Re-confirm emptiness against a fresh tail_ first: the snapshot
// at the top of the loop may be stale (the producer can push more
// values *and* the sentinel in the window since), and the sentinel
// must never jump ahead of ring values pushed before it. The spin
// and post-spin takes below already reload tail_ on the line above
// them; this is the one take that used the loop-top snapshot.
if (h == tail_.load(std::memory_order_acquire)) {
T s; if (take_sentinel(s)) return s;
}
if (!accepting_.load(std::memory_order_acquire))
throw ChannelClosedError{};
for (std::size_t si = 0; si < spin_count_; ++si) {
spin_hint();
t = tail_.load(std::memory_order_acquire);
if (t != h) break;
{ T s; if (take_sentinel(s)) return s; }
if (!accepting_.load(std::memory_order_relaxed))
throw ChannelClosedError{};
}
if (h == t) {
// Still empty after spin — sleep until push()/push_sentinel()
// or disable() fires. Re-check tail and the sentinel after
// loading w to guard against a lost wakeup.
if (tail_.load(std::memory_order_acquire) != h) continue;
if (has_eof_.load(std::memory_order_acquire)) continue;
wake_.wait(w, std::memory_order_relaxed);
continue;
}
}
// Item available (found immediately or during spin).
if (!accepting_.load(std::memory_order_acquire))
throw ChannelClosedError{};
T value = extract(std::move(buf_[h & ring_mask_]));
head_.store(h + 1, std::memory_order_release);
// A slot just freed: wake any producer parked on this channel.
if (t - h >= capacity_ && space_callback_) space_callback_();
stats_.record_pop();
return value;
}
}
// Non-blocking pop with timeout. For watchdog/display use only.
bool try_pop(T& out, std::chrono::milliseconds timeout) {
const auto deadline = std::chrono::steady_clock::now() + timeout;
for (;;) {
if (try_pop_now(out)) return true;
if (!accepting_.load(std::memory_order_relaxed)) return false;
if (std::chrono::steady_clock::now() >= deadline) return false;
std::this_thread::sleep_for(std::chrono::microseconds(50));
}
}
// Immediate non-blocking pop. Returns false if the ring is empty.
// Once the ring is drained, delivers any pending out-of-band sentinel (EOF)
// so pool nodes — which pop only via this path — still receive the token.
bool try_pop_now(T& out) {
const std::size_t h = head_.load(std::memory_order_relaxed);
const std::size_t t = tail_.load(std::memory_order_acquire);
if (h == t)
return take_sentinel(out);
out = extract(std::move(buf_[h & ring_mask_]));
head_.store(h + 1, std::memory_order_release);
stats_.record_pop();
// Pool nodes pop only through here, so this is where a parked producer
// gets woken: the ring was full, and it no longer is.
if (t - h >= capacity_ && space_callback_) space_callback_();
return true;
}
// Enable the channel (called by consumer node on start()).
void enable() {
accepting_.store(true, std::memory_order_relaxed);
}
// Disable the channel: stop accepting new pushes, unblock any waiting pop().
// Items already in the ring are abandoned and freed when the Channel is destroyed.
void disable() {
accepting_.store(false, std::memory_order_release);
wake_.fetch_add(1, std::memory_order_release);
wake_.notify_all();
}
// Register a callback fired when the queue transitions empty→non-empty.
void set_push_callback(std::function<void()> cb) {
push_callback_ = std::move(cb);
}
// Ring occupancy, derived lazily from indices — no separate counter on the
// hot path. Excludes any out-of-band sentinel (that lives outside the ring).
std::size_t size() const {
return tail_.load(std::memory_order_relaxed)
- head_.load(std::memory_order_relaxed);
}
// A pending out-of-band sentinel (EOF) counts as consumable work here even
// though it holds no ring slot. This is what node readiness checks call, so
// a channel carrying only a sentinel still schedules its consumer's next
// fire — without this the sentinel would never be popped and the pipeline
// would deadlock at teardown.
std::size_t approx_size() const {
return size() + (has_eof_.load(std::memory_order_acquire) ? 1u : 0u);
}
std::size_t capacity() const { return capacity_; }
bool is_accepting() const { return accepting_.load(std::memory_order_relaxed); }
const ChannelStats& stats() const { return stats_; }
ChannelSnapshot snapshot(const std::string& name) const {
const std::size_t t = tail_.load(std::memory_order_relaxed);
const std::size_t h = head_.load(std::memory_order_relaxed);
return {
name,
capacity_,
t - h,
stats_.peak_fill.load(std::memory_order_relaxed),
stats_.pushes.load(std::memory_order_relaxed),
stats_.bytes_pushed.load(std::memory_order_relaxed),
stats_.drops.load(std::memory_order_relaxed),
stats_.overflows.load(std::memory_order_relaxed),
stats_.pops.load(std::memory_order_relaxed),
sizeof(T),
};
}
private:
static storage_type make_storage(T&& v) {
if constexpr (channel_storage_policy<T>::by_value)
return std::move(v);
else
return std::make_shared<const T>(std::move(v));
}
static T extract(storage_type&& s) {
if constexpr (channel_storage_policy<T>::by_value)
return std::move(s);
else
return *s;
}
// Consume the out-of-band sentinel if one is pending. Consumer-only.
// Called only when the ring is observed empty, so the sentinel is always
// delivered after every value pushed before it.
bool take_sentinel(T& out) {
if (!has_eof_.load(std::memory_order_acquire)) return false;
out = extract(std::move(eof_value_));
has_eof_.store(false, std::memory_order_release);
stats_.record_pop();
return true;
}
const std::size_t capacity_;
const std::size_t spin_count_;
std::size_t ring_mask_;
std::unique_ptr<storage_type[]> buf_;
std::function<void()> push_callback_;
std::function<void()> space_callback_;
ChannelStats stats_;
// Out-of-band sentinel (EOF): stored outside the ring so its delivery never
// depends on ring capacity and never blocks the producer. Written by the
// producer (push_sentinel), read+cleared by the consumer (take_sentinel);
// has_eof_ is the publish/consume handshake.
storage_type eof_value_{};
std::atomic<bool> has_eof_{false};
// Separate cache lines: head_ is written only by the consumer;
// tail_ and wake_ are written only by the producer.
// wake_ wakes a blocked pop() on enqueue or on a pending sentinel.
alignas(64) std::atomic<std::size_t> head_{0};
alignas(64) std::atomic<std::size_t> tail_{0};
std::atomic<uint32_t> wake_{0};
std::atomic<bool> accepting_{true};
};
// ── Channel probe — type-erased snapshot accessor ─────────────────────────────
// Used by both Network and StaticNetwork for diagnostics.
struct IChannelProbe {
virtual ~IChannelProbe() = default;
virtual ChannelSnapshot snapshot() const = 0;
};
template<typename T>
struct ChannelProbe : IChannelProbe {
const Channel<T>& ch;
std::string name;
ChannelProbe(const Channel<T>& c, std::string n) : ch(c), name(std::move(n)) {}
ChannelSnapshot snapshot() const override { return ch.snapshot(name); }
};
} // namespace kpn
+456
View File
@@ -0,0 +1,456 @@
#pragma once
// Only active when KPN_WEB_DEBUG is defined.
#ifdef KPN_WEB_DEBUG
#include "diagnostics.hpp"
#include "web_debug.hpp"
#include <functional>
#include <memory>
#include <sstream>
#include <string>
#include <utility>
#include <vector>
namespace kpn {
// ── Hub HTML ──────────────────────────────────────────────────────────────────
// Multi-tab UI: one tab per registered network (force-directed graph) +
// an "All Networks" tab showing shared resource cards and a cross-network
// node table.
static const char* HUB_HTML = R"html(<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>KPN++ Debug Hub</title>
<style>
*{box-sizing:border-box}
body{margin:0;background:#1a1a2e;color:#eee;font-family:monospace}
#hdr{display:flex;align-items:center;padding:0 16px;background:#16213e;
border-bottom:1px solid #0f3460;height:44px;gap:8px;overflow-x:auto}
#hdr h1{margin:0;font-size:16px;color:#e94560;white-space:nowrap;margin-right:12px}
#tab-bar{display:flex;gap:2px;flex:1}
.tab{padding:4px 14px;border:none;background:#0f3460;color:#aaa;
cursor:pointer;font-family:monospace;font-size:11px;border-radius:2px;white-space:nowrap}
.tab.active{background:#e94560;color:#fff}
.tab:hover:not(.active){background:#1e3a6e;color:#eee}
#status{font-size:10px;color:#555;white-space:nowrap}
.panel{display:none}
.panel.active{display:block}
/* ── All Networks tab ─────────────────────────────────────── */
#panel-all{height:calc(100vh - 44px);overflow-y:auto;padding:16px;
display:none;gap:16px;grid-template-columns:300px 1fr}
#panel-all.active{display:grid;align-content:start}
#panel-all h2{font-size:11px;color:#e94560;margin:0 0 8px;
letter-spacing:1px;text-transform:uppercase}
#res-col{grid-column:1}
.res-card{background:#16213e;border:1px solid #0f3460;border-radius:4px;
padding:10px 12px;margin-bottom:8px}
.res-head{display:flex;justify-content:space-between;align-items:center}
.res-name{font-size:12px}
.badge{font-size:9px;padding:2px 6px;border-radius:2px}
.held{background:#e94560}.free{background:#4CAF50;color:#111}
.res-meta{font-size:9px;color:#666;margin-top:5px;display:flex;gap:12px;flex-wrap:wrap}
.bar-wrap{height:3px;background:#0f3460;border-radius:2px;margin-top:7px}
.bar{height:3px;border-radius:2px;transition:width 0.4s}
#nodes-col{grid-column:2;overflow-y:auto;max-height:calc(100vh - 76px)}
table{width:100%;border-collapse:collapse;font-size:10px}
th{padding:4px 8px;color:#555;border-bottom:1px solid #0f3460;text-align:left;
position:sticky;top:0;background:#1a1a2e;z-index:1}
td{padding:2px 8px;border-bottom:1px solid #16213e}
tr:hover td{background:#16213e}
.ntag{font-size:9px;background:#0f3460;padding:1px 4px;border-radius:2px;color:#4CAF50}
/* ── Per-network graph panels ─────────────────────────────── */
.graph-panel{width:100vw;height:calc(100vh - 44px)}
svg.net{width:100%;height:100%}
.node circle{stroke:#fff;stroke-width:1.5px}
.node text{font-size:11px;fill:#eee;pointer-events:none;text-anchor:middle}
.node .st{font-size:9px;fill:#aaa}
.link{fill:none;stroke-width:2px}
.lbl{font-size:9px;fill:#ccc}
#tip{position:absolute;background:#0f3460;border:1px solid #e94560;border-radius:4px;
padding:8px 12px;font-size:11px;pointer-events:none;display:none;
white-space:pre;line-height:1.6}
</style>
</head>
<body>
<div id="hdr">
<h1>KPN++ Debug Hub</h1>
<div id="tab-bar"></div>
<span id="status">connecting</span>
</div>
<div id="panels">
<div id="panel-all" class="panel"></div>
</div>
<div id="tip"></div>
<script src="https://d3js.org/d3.v7.min.js"></script>
<script>
const R = 28;
const tip = d3.select('#tip');
const nc = ema => ema>100?'#e94560':ema>50?'#e07040':ema>10?'#f0c040':'#4CAF50';
const ec = pct => pct>=80?'#e94560':pct>=50?'#f0c040':'#4CAF50';
const ea = pct => pct>=80?'url(#a2)':pct>=50?'url(#a1)':'url(#a0)';
// ── Tab management ────────────────────────────────────────────────────────────
let activeTab = null;
function ensureTab(id, label) {
if (document.getElementById('tab-' + id)) return;
const b = document.createElement('button');
b.className = 'tab'; b.id = 'tab-' + id; b.textContent = label;
b.onclick = () => showTab(id);
document.getElementById('tab-bar').appendChild(b);
}
function showTab(id) {
activeTab = id;
document.querySelectorAll('.tab').forEach(b =>
b.classList.toggle('active', b.id === 'tab-' + id));
document.querySelectorAll('.panel').forEach(p =>
p.classList.toggle('active', p.id === 'panel-' + id));
}
// ── All Networks tab ──────────────────────────────────────────────────────────
function renderAll(data) {
const panel = document.getElementById('panel-all');
// Resources column
let rhtml = '<div id="res-col"><h2>Shared Resources</h2>';
if (!data.resources || !data.resources.length)
rhtml += '<div style="color:#444;font-size:11px">None registered</div>';
for (const r of (data.resources || [])) {
const wpct = Math.min(100, r.avg_wait_ms).toFixed(1);
const bc = r.current_waiters > 0 ? '#e94560' : '#4CAF50';
rhtml += `<div class="res-card">
<div class="res-head">
<span class="res-name">${r.name}</span>
<span class="badge ${r.held ? 'held' : 'free'}">${r.held ? 'HELD' : 'free'}</span>
</div>
<div class="res-meta">
<span>avg wait ${r.avg_wait_ms.toFixed(1)} ms</span>
<span>waiters ${r.current_waiters} / peak ${r.peak_waiters}</span>
<span>${r.acquisitions} acq</span>
</div>
<div class="bar-wrap">
<div class="bar" style="width:${wpct}%;background:${bc}"></div>
</div>
</div>`;
}
rhtml += '</div>';
// Nodes column — all networks in one table
let rows = '';
for (const net of data.networks) {
for (const n of net.nodes) {
rows += `<tr>
<td><span class="ntag">${net.name}</span></td>
<td>${n.id}</td>
<td>${n.fps.toFixed(1)}</td>
<td>${n.ema_exec_ms.toFixed(2)}</td>
<td>${n.max_exec_ms.toFixed(2)}</td>
<td>${n.blocked_ms.toFixed(2)}</td>
<td>${n.cpu_util_pct.toFixed(1)}</td>
</tr>`;
}
}
const thtml = `<div id="nodes-col"><h2>All Nodes</h2>
<table>
<tr><th>Network</th><th>Node</th><th>fps</th>
<th>exec ema (ms)</th><th>exec max (ms)</th>
<th>blocked (ms)</th><th>cpu %</th></tr>
${rows}
</table></div>`;
panel.innerHTML = rhtml + thtml;
}
// ── Per-network graph ─────────────────────────────────────────────────────────
const nets = {};
function initNet(netData) {
const name = netData.name;
const div = document.createElement('div');
div.id = 'panel-' + name;
div.className = 'panel graph-panel';
document.getElementById('panels').appendChild(div);
const svg = d3.select(div).append('svg').attr('class', 'net');
const W = () => div.clientWidth || window.innerWidth;
const H = () => div.clientHeight || (window.innerHeight - 44);
const defs = svg.append('defs');
['#4CAF50','#f0c040','#e94560'].forEach((col, i) =>
defs.append('marker').attr('id','a'+i)
.attr('viewBox','0 -5 10 10').attr('refX',10).attr('refY',0)
.attr('markerWidth',6).attr('markerHeight',6).attr('orient','auto')
.append('path').attr('d','M0,-5L10,0L0,5').attr('fill',col));
const g = svg.append('g');
svg.call(d3.zoom().on('zoom', e => g.attr('transform', e.transform)));
const nodes = netData.nodes.map(n => ({...n, x: W()/2, y: H()/2}));
const byId = Object.fromEntries(nodes.map(n => [n.id, n]));
const links = netData.edges
.map(e => ({...e, source: byId[e.source], target: byId[e.target]}))
.filter(e => e.source && e.target);
const sim = d3.forceSimulation(nodes)
.force('link', d3.forceLink(links).distance(150).strength(0.5))
.force('charge', d3.forceManyBody().strength(-350))
.force('center', d3.forceCenter(W()/2, H()/2))
.force('collide', d3.forceCollide(R + 18))
.on('tick', tick);
const lsel = g.append('g').selectAll('line').data(links).join('line')
.attr('class','link')
.attr('stroke', d => ec(d.fill_pct))
.attr('marker-end', d => ea(d.fill_pct));
const llbl = g.append('g').selectAll('text').data(links).join('text')
.attr('class','lbl').text(d => `${d.fill_pct.toFixed(0)}%`);
const ng = g.append('g').selectAll('g').data(nodes).join('g').attr('class','node')
.call(d3.drag()
.on('start',(e,d)=>{ if(!e.active) sim.alphaTarget(0.3).restart(); d.fx=d.x; d.fy=d.y; })
.on('drag', (e,d)=>{ d.fx=e.x; d.fy=e.y; })
.on('end', (e,d)=>{ if(!e.active) sim.alphaTarget(0); d.fx=null; d.fy=null; }));
ng.append('circle').attr('r', R).attr('fill', d => nc(d.ema_exec_ms));
ng.append('text').attr('dy', 4).text(d => d.id);
ng.append('text').attr('class','st').attr('dy', 18)
.text(d => `${d.ema_exec_ms.toFixed(1)}ms ${d.fps.toFixed(1)}fps`);
ng.on('mousemove', (e,d) =>
tip.style('display','block')
.style('left',(e.pageX+12)+'px').style('top',(e.pageY+12)+'px')
.text(`${d.id}\nframes: ${d.frames} fps: ${d.fps.toFixed(2)}\n` +
`exec ema: ${d.ema_exec_ms.toFixed(2)}ms max: ${d.max_exec_ms.toFixed(2)}ms\n` +
`blocked: ${d.blocked_ms.toFixed(2)}ms cpu: ${d.cpu_util_pct.toFixed(1)}%`))
.on('mouseleave', () => tip.style('display','none'));
g.selectAll('.link')
.on('mousemove', (e,d) =>
tip.style('display','block')
.style('left',(e.pageX+12)+'px').style('top',(e.pageY+12)+'px')
.text(`${d.name}\nfill: ${d.fill_pct.toFixed(1)}% peak: ${d.peak_pct.toFixed(1)}%\n` +
`cap: ${d.capacity} pushes: ${d.pushes} drops: ${d.drops}\n` +
`bandwidth: ${(d.bw_mbs||0).toFixed(2)} MB/s`))
.on('mouseleave', () => tip.style('display','none'));
function tick() {
const w = W(), h = H();
nodes.forEach(d => {
d.x = Math.max(R, Math.min(w - R, d.x));
d.y = Math.max(R, Math.min(h - R, d.y));
});
lsel
.attr('x1', d => d.source.x).attr('y1', d => d.source.y)
.attr('x2', d => { const dx=d.target.x-d.source.x, dy=d.target.y-d.source.y,
dist=Math.sqrt(dx*dx+dy*dy)||1;
return d.target.x-(dx/dist)*(R+8); })
.attr('y2', d => { const dx=d.target.x-d.source.x, dy=d.target.y-d.source.y,
dist=Math.sqrt(dx*dx+dy*dy)||1;
return d.target.y-(dy/dist)*(R+8); });
llbl.attr('x', d => (d.source.x+d.target.x)/2)
.attr('y', d => (d.source.y+d.target.y)/2 - 6);
ng.attr('transform', d => `translate(${d.x},${d.y})`);
}
nets[name] = {nodes, links, sim, ng, lsel, llbl};
}
function updateNet(netData) {
const st = nets[netData.name];
if (!st) return;
const byId = Object.fromEntries(netData.nodes.map(n => [n.id, n]));
st.nodes.forEach(n => {
const f = byId[n.id];
if (f) Object.assign(n, {frames:f.frames, ema_exec_ms:f.ema_exec_ms,
max_exec_ms:f.max_exec_ms, blocked_ms:f.blocked_ms, fps:f.fps,
total_cpu_ms:f.total_cpu_ms, cpu_util_pct:f.cpu_util_pct});
});
netData.edges.forEach((e,i) => {
if (st.links[i]) Object.assign(st.links[i], {fill_pct:e.fill_pct,
peak_pct:e.peak_pct, pushes:e.pushes, drops:e.drops,
overflows:e.overflows, current:e.current, bw_mbs:e.bw_mbs});
});
st.ng.select('circle').attr('fill', d => nc(d.ema_exec_ms));
st.ng.select('.st').text(d => `${d.ema_exec_ms.toFixed(1)}ms ${d.fps.toFixed(1)}fps`);
st.lsel.attr('stroke', d => ec(d.fill_pct)).attr('marker-end', d => ea(d.fill_pct));
st.llbl.text(d => `${d.fill_pct.toFixed(0)}%`);
}
// ── Poll loop ─────────────────────────────────────────────────────────────────
let init = false;
async function poll() {
try {
const r = await fetch('/api/snapshot');
if (!r.ok) throw new Error(r.status);
const data = await r.json();
if (!init) {
ensureTab('all', 'All Networks');
data.networks.forEach(net => ensureTab(net.name, net.name));
showTab('all');
data.networks.forEach(initNet);
init = true;
}
renderAll(data);
data.networks.forEach(updateNet);
document.getElementById('status').textContent =
`${new Date().toLocaleTimeString()} · ${data.networks.length} nets · ${(data.resources||[]).length} resources`;
} catch(e) {
document.getElementById('status').textContent = 'error: ' + e;
}
}
poll();
setInterval(poll, 500);
window.addEventListener('resize', () =>
Object.values(nets).forEach(st =>
st.sim.force('center', d3.forceCenter(
(document.getElementById('panel-' + Object.keys(nets).find(k => nets[k] === st))?.clientWidth || window.innerWidth) / 2,
(document.getElementById('panel-' + Object.keys(nets).find(k => nets[k] === st))?.clientHeight || window.innerHeight - 44) / 2
)).alpha(0.1).restart()));
</script>
</body>
</html>
)html";
// ── DebugHub ──────────────────────────────────────────────────────────────────
class DebugHub {
public:
explicit DebugHub(uint16_t port = 9090) : port_(port) {}
~DebugHub() { stop(); }
DebugHub(const DebugHub&) = delete;
DebugHub& operator=(const DebugHub&) = delete;
// Register a network. Disables that network's own web server so the hub
// becomes the single debug endpoint. Call before network.start().
template<typename Net>
void register_network(const std::string& name, Net& net) {
net.disable_web_server();
networks_.push_back({name, [&net, name] {
auto s = net.network_snapshot();
s.name = name;
return s;
}});
}
// Register a shared resource — appears in the "All Networks" resource panel.
void register_resource(const std::string& name, IResourceProbe* probe) {
resources_.emplace_back(name, probe);
}
void start() {
server_ = std::make_unique<web_debug::WebDebugServer>(
port_,
[this] { return build_json(); },
HUB_HTML);
server_->start();
std::cerr << "[kpn] hub debug UI: http://localhost:" << port_ << "\n";
}
void stop() { if (server_) server_->stop(); }
private:
// Serialise nodes array for one network snapshot
static void write_nodes(std::ostream& o, const NetworkSnapshot& s) {
o << "[";
for (std::size_t i = 0; i < s.nodes.size(); ++i) {
const auto& n = s.nodes[i];
if (i) o << ',';
o << "{\"id\":\"" << web_debug::escape_json(n.name) << "\""
<< ",\"frames\":" << n.frames_processed
<< ",\"ema_exec_ms\":" << n.ema_exec_ms
<< ",\"max_exec_ms\":" << n.max_exec_ms
<< ",\"blocked_ms\":" << n.total_blocked_ms
<< ",\"fps\":" << n.throughput_fps
<< ",\"total_cpu_ms\":" << n.total_cpu_ms
<< ",\"cpu_util_pct\":" << n.cpu_util_pct
<< "}";
}
o << "]";
}
// Serialise edges array for one network snapshot
static void write_edges(std::ostream& o, const NetworkSnapshot& s) {
o << "[";
for (std::size_t i = 0; i < s.channels.size(); ++i) {
const auto& c = s.channels[i];
if (i) o << ',';
auto [src, dst] = web_debug::parse_edge_name(c.name);
o << "{\"name\":\"" << web_debug::escape_json(c.name) << "\""
<< ",\"source\":\"" << web_debug::escape_json(src) << "\""
<< ",\"target\":\"" << web_debug::escape_json(dst) << "\""
<< ",\"capacity\":" << c.capacity
<< ",\"current\":" << c.current_fill
<< ",\"fill_pct\":" << c.fill_pct()
<< ",\"peak_pct\":" << c.peak_pct()
<< ",\"pushes\":" << c.pushes
<< ",\"drops\":" << c.drops
<< ",\"overflows\":" << c.overflows
<< ",\"item_bytes\":" << c.item_bytes
<< ",\"bw_mbs\":" << c.bandwidth_mbs(s.elapsed_s)
<< "}";
}
o << "]";
}
std::string build_json() const {
std::ostringstream o;
o << std::fixed;
o.precision(2);
o << "{\"networks\":[";
for (std::size_t i = 0; i < networks_.size(); ++i) {
if (i) o << ',';
auto s = networks_[i].fn();
o << "{\"name\":\"" << web_debug::escape_json(networks_[i].name) << "\""
<< ",\"nodes\":"; write_nodes(o, s);
o << ",\"edges\":"; write_edges(o, s);
o << "}";
}
o << "],\"resources\":[";
for (std::size_t i = 0; i < resources_.size(); ++i) {
if (i) o << ',';
const auto r = resources_[i].second->snapshot(resources_[i].first);
o << "{\"name\":\"" << web_debug::escape_json(r.name) << "\""
<< ",\"acquisitions\":" << r.acquisitions
<< ",\"avg_wait_ms\":" << r.avg_wait_ms
<< ",\"peak_waiters\":" << r.peak_waiters
<< ",\"current_waiters\":" << r.current_waiters
<< ",\"held\":" << (r.held ? "true" : "false")
<< "}";
}
o << "]}";
return o.str();
}
struct Entry {
std::string name;
std::function<NetworkSnapshot()> fn;
};
uint16_t port_;
std::vector<Entry> networks_;
std::vector<std::pair<std::string,IResourceProbe*>> resources_;
std::unique_ptr<web_debug::WebDebugServer> server_;
};
} // namespace kpn
#endif // KPN_WEB_DEBUG
+193
View File
@@ -0,0 +1,193 @@
#pragma once
#include <atomic>
#include <chrono>
#include <cstddef>
#include <cstdint>
#include <string>
#include <time.h> // clock_gettime, CLOCK_THREAD_CPUTIME_ID
namespace kpn {
using clock_t = std::chrono::steady_clock;
using duration_t = std::chrono::duration<double, std::milli>; // milliseconds
// ── Per-channel statistics ────────────────────────────────────────────────────
struct ChannelStats {
std::atomic<uint64_t> pushes{0};
std::atomic<uint64_t> bytes_pushed{0};
std::atomic<uint64_t> drops{0};
std::atomic<uint64_t> overflows{0};
std::atomic<uint64_t> pops{0};
std::atomic<std::size_t> peak_fill{0};
ChannelStats() = default;
ChannelStats(const ChannelStats&) = delete;
ChannelStats& operator=(const ChannelStats&) = delete;
void record_push(std::size_t current_fill, std::size_t data_bytes) {
pushes.fetch_add(1, std::memory_order_relaxed);
bytes_pushed.fetch_add(data_bytes, std::memory_order_relaxed);
std::size_t prev = peak_fill.load(std::memory_order_relaxed);
while (current_fill > prev &&
!peak_fill.compare_exchange_weak(prev, current_fill,
std::memory_order_relaxed, std::memory_order_relaxed))
;
}
void record_drop() { drops.fetch_add(1, std::memory_order_relaxed); }
void record_overflow() { overflows.fetch_add(1, std::memory_order_relaxed); }
void record_pop() { pops.fetch_add(1, std::memory_order_relaxed); }
};
// ── Per-node statistics ───────────────────────────────────────────────────────
struct NodeStats {
std::atomic<uint64_t> frames_processed{0};
// Wall-clock execution time EMA — warmup mean for first WARMUP_FRAMES,
// then EMA alpha=0.1. Stored as integer microseconds for atomic updates.
static constexpr int WARMUP_FRAMES = 5;
std::atomic<int64_t> ema_exec_us{0};
std::atomic<int64_t> max_exec_us{0};
std::atomic<int64_t> total_blocked_us{0};
// Thread CPU time — actual CPU consumed by this node's thread,
// measured via CLOCK_THREAD_CPUTIME_ID. Excludes time sleeping or
// blocked on mutexes/channels. Sampled once per frame.
std::atomic<int64_t> total_cpu_us{0}; // cumulative CPU µs consumed
// Pool scheduling stats — only meaningful for PoolNode / InterruptNode.
// exec_start_us: wall-clock µs when fire_once began; 0 when idle.
// Used by the watchdog to detect hung nodes (elapsed > max_exec_time).
std::atomic<int64_t> queue_wait_us{0}; // cumulative µs spent in pool queue
std::atomic<int64_t> exec_start_us{0}; // non-zero while fire_once is running
NodeStats() = default;
NodeStats(const NodeStats&) = delete;
NodeStats& operator=(const NodeStats&) = delete;
// Call at the start of run_loop to capture thread CPU baseline.
// Returns the raw timespec for use in record_exec.
static struct timespec cpu_now() {
struct timespec ts{};
clock_gettime(CLOCK_THREAD_CPUTIME_ID, &ts);
return ts;
}
static int64_t timespec_us(const struct timespec& ts) {
return static_cast<int64_t>(ts.tv_sec) * 1'000'000
+ static_cast<int64_t>(ts.tv_nsec) / 1'000;
}
void record_queue_wait(duration_t wait) {
int64_t us = static_cast<int64_t>(wait.count() * 1000.0);
if (us > 0) queue_wait_us.fetch_add(us, std::memory_order_relaxed);
}
void record_exec(duration_t exec_time, duration_t blocked_time,
const struct timespec& cpu_before, const struct timespec& cpu_after) {
frames_processed.fetch_add(1, std::memory_order_relaxed);
int64_t us = static_cast<int64_t>(exec_time.count() * 1000.0);
uint64_t n = frames_processed.load(std::memory_order_relaxed);
int64_t prev = ema_exec_us.load(std::memory_order_relaxed);
int64_t next = (n <= static_cast<uint64_t>(WARMUP_FRAMES))
? prev + (us - prev) / static_cast<int64_t>(n)
: prev + (us - prev) / 10;
ema_exec_us.store(next, std::memory_order_relaxed);
int64_t cur_max = max_exec_us.load(std::memory_order_relaxed);
if (us > cur_max)
max_exec_us.store(us, std::memory_order_relaxed);
int64_t blocked_us = static_cast<int64_t>(blocked_time.count() * 1000.0);
total_blocked_us.fetch_add(blocked_us, std::memory_order_relaxed);
int64_t cpu_delta = timespec_us(cpu_after) - timespec_us(cpu_before);
if (cpu_delta > 0)
total_cpu_us.fetch_add(cpu_delta, std::memory_order_relaxed);
}
};
// ── Snapshot for reporting (copyable, taken by watchdog) ─────────────────────
struct ChannelSnapshot {
std::string name;
std::size_t capacity;
std::size_t current_fill;
std::size_t peak_fill;
uint64_t pushes;
uint64_t bytes_pushed; // actual bytes accumulated via channel_data_size<T>
uint64_t drops;
uint64_t overflows;
uint64_t pops;
std::size_t item_bytes; // sizeof(T) — nominal struct size, not necessarily data size
double fill_pct() const {
return capacity ? 100.0 * current_fill / capacity : 0.0;
}
double peak_pct() const {
return capacity ? 100.0 * peak_fill / capacity : 0.0;
}
// Bandwidth in MB/s: actual bytes transferred / elapsed seconds
double bandwidth_mbs(double elapsed_s) const {
if (elapsed_s <= 0.0) return 0.0;
return static_cast<double>(bytes_pushed) / elapsed_s / 1e6;
}
};
struct NodeSnapshot {
std::string name;
uint64_t frames_processed;
double ema_exec_ms;
double max_exec_ms;
double total_blocked_ms; // ThreadPerNode: time blocked in channel pop
double throughput_fps;
double total_cpu_ms; // cumulative CPU time consumed by this node's thread
double cpu_util_pct; // exec_ms / (exec_ms + blocked_ms) * 100
double queue_wait_ms{0}; // PoolNode: cumulative time spent in pool queue
};
// ── Pool statistics + snapshot ────────────────────────────────────────────────
struct PoolSnapshot {
std::string name;
std::size_t thread_count;
std::size_t queue_depth; // tasks waiting in the priority queue
std::size_t active_count; // tasks currently executing
uint64_t tasks_submitted;
uint64_t tasks_completed;
};
struct IPoolProbe {
virtual ~IPoolProbe() = default;
virtual PoolSnapshot snapshot(const std::string& name) const = 0;
};
// ── Cross-network snapshot (used by DebugHub) ─────────────────────────────────
struct NetworkSnapshot {
std::string name;
std::vector<NodeSnapshot> nodes;
std::vector<ChannelSnapshot> channels;
double elapsed_s;
};
// ── Resource statistics + snapshot ───────────────────────────────────────────
struct ResourceSnapshot {
std::string name;
uint64_t acquisitions;
double avg_wait_ms;
uint64_t peak_waiters;
uint64_t current_waiters;
bool held;
};
struct IResourceProbe {
virtual ~IResourceProbe() = default;
virtual ResourceSnapshot snapshot(const std::string& name) const = 0;
};
} // namespace kpn
+159
View File
@@ -0,0 +1,159 @@
#pragma once
#include "channel.hpp"
#include "diagnostics.hpp"
#include "inode.hpp"
#include "port.hpp"
#include "traits.hpp"
#include <array>
#include <atomic>
#include <iostream>
#include <memory>
#include <thread>
#include <tuple>
#include <utility>
namespace kpn {
// ── FanoutNode ────────────────────────────────────────────────────────────────
//
// Reads one item from its single input channel and pushes a copy to each of
// N output channels. All N downstream nodes receive every item.
//
// Usage:
// auto fan = make_fanout<Image, 2>(/*capacity=*/8);
// net.connect("src", src.output<0>(), "fan", fan.input<0>())
// .connect("fan", fan.output<0>(), "nodeA", nodeA.input<0>())
// .connect("fan", fan.output<1>(), "nodeB", nodeB.input<0>())
template<typename T, std::size_t N, std::size_t Id = 0>
class FanoutNode : public INode {
public:
using args_tuple = std::tuple<T>;
using return_tuple = repeat_tuple_t<T, N>;
using return_raw = return_tuple;
static constexpr std::size_t input_count = 1;
static constexpr std::size_t output_count = N;
static constexpr std::size_t unique_tag = Id;
static constexpr bool is_fanout_node = true;
explicit FanoutNode(std::size_t fifo_capacity = 5)
: fifo_capacity_(fifo_capacity)
{
input_ch_ = std::make_shared<Channel<T>>(fifo_capacity);
}
~FanoutNode() override { stop(); }
// ── INode ─────────────────────────────────────────────────────────────────
void start() override {
input_ch_->enable();
stop_flag_.store(false, std::memory_order_relaxed);
thread_ = std::jthread([this](std::stop_token) { run_loop(); });
}
void stop() override {
stop_flag_.store(true, std::memory_order_relaxed);
input_ch_->disable();
if (thread_.joinable()) thread_.request_stop(), thread_.join();
}
bool running() const override {
return thread_.joinable() && !stop_flag_.load(std::memory_order_relaxed);
}
void set_name(std::string name) override { name_ = std::move(name); }
const NodeStats& stats() const override { return stats_; }
NodeSnapshot node_snapshot(const std::string& name, double elapsed_s) const override {
uint64_t frames = stats_.frames_processed.load(std::memory_order_relaxed);
double exec_ms = stats_.ema_exec_us.load(std::memory_order_relaxed) / 1000.0;
double blocked_ms = stats_.total_blocked_us.load(std::memory_order_relaxed) / 1000.0;
double total_ms = exec_ms + blocked_ms;
return {name, frames, exec_ms,
stats_.max_exec_us.load(std::memory_order_relaxed) / 1000.0,
blocked_ms,
elapsed_s > 0 ? frames / elapsed_s : 0.0,
stats_.total_cpu_us.load(std::memory_order_relaxed) / 1000.0,
total_ms > 0 ? 100.0 * exec_ms / total_ms : 0.0};
}
// ── Port access ───────────────────────────────────────────────────────────
template<std::size_t I = 0>
InputPort<FanoutNode, I> input() {
static_assert(I == 0, "FanoutNode has exactly one input");
return {*this};
}
template<std::size_t I>
OutputPort<FanoutNode, I> output() {
static_assert(I < N, "FanoutNode output index out of range");
return {*this};
}
// ── Internal channel accessors (called by Network::connect) ───────────────
template<std::size_t I>
Channel<T>& input_channel() {
static_assert(I == 0);
return *input_ch_;
}
template<std::size_t I>
void set_input_channel(std::shared_ptr<Channel<T>> ch) {
static_assert(I == 0);
input_ch_ = std::move(ch);
}
template<std::size_t I>
void set_output_channel(Channel<T>* ch) {
static_assert(I < N);
out_channels_[I] = ch;
}
private:
void run_loop() {
while (!stop_flag_.load(std::memory_order_relaxed)) {
try {
auto t0 = clock_t::now();
T val = input_ch_->pop();
auto t1 = clock_t::now();
auto cpu0 = NodeStats::cpu_now();
for (std::size_t i = 0; i < N; ++i) {
if (out_channels_[i]) {
try { out_channels_[i]->push(val); }
catch (const ChannelOverflowError&) {} // drop for this output independently
}
}
auto cpu1 = NodeStats::cpu_now();
auto t2 = clock_t::now();
stats_.record_exec(duration_t(t2 - t1), duration_t(t1 - t0), cpu0, cpu1);
} catch (const ChannelClosedError&) {
break;
}
}
}
std::string name_;
std::size_t fifo_capacity_;
std::shared_ptr<Channel<T>> input_ch_;
std::array<Channel<T>*, N> out_channels_{};
std::atomic<bool> stop_flag_{false};
std::jthread thread_;
NodeStats stats_;
};
// ── Factory ───────────────────────────────────────────────────────────────────
template<typename T, std::size_t N>
FanoutNode<T, N> make_fanout(std::size_t fifo_capacity = 5) {
return FanoutNode<T, N, 0>(fifo_capacity);
}
} // namespace kpn

Some files were not shown because too many files have changed in this diff Show More