Files
KPN/include/kpn/static_network.hpp
T
dtourolle 28e06675f5
🚦 CI / changes (push) Successful in 6s
🚦 CI / docker (push) Has been skipped
🚦 CI / test (push) Failing after 4m20s
🚦 CI / tsan (push) Failing after 3m16s
🚦 CI / docs (push) Has been skipped
fix: park nodes on a full output instead of blocking the worker
push_blocking parked a scheduler worker inside the push. Nodes own a
private single-thread pool, so the parked thread was the only one that
could drain that node's own input — hold-and-wait, and under sustained
backpressure four nodes of a five-node chain slept in nanosleep at once.
channel.hpp already warned about this for sentinels; it applies just as
much to data pushes.

The scheduler was purely input-driven: on_input_ready() wakes a node when
input arrives, with no counterpart for "my output has room". Lacking that
signal, blocking the thread was the only way to handle a full output.
This adds the missing half.

- Channel::try_push + has_space + set_space_callback; the callback fires
  from both pop() and try_pop_now().
- PoolNode/PoolObjectNode keep a one-slot pending_ buffer with per-element
  done flags, so a retry cannot duplicate an already-accepted element. One
  slot suffices because queued_ admits at most one fire_once per node.
- The re-check after clearing queued_ closes the lost-wakeup race where a
  space callback fires while the flag is still up and is swallowed.

Two bugs surfaced once nodes actually parked, both fixed here:

- pop_one reports an *empty* channel as ChannelClosedError, which is also
  the node's "upstream finished, self-stop" signal. A node woken by output
  space with empty inputs therefore killed itself. fire_once now releases
  the worker when its inputs are not ready rather than falling through.
- The drained-park path resubmitted unconditionally instead of via
  on_input_ready(), firing nodes with nothing to read.

compute_priority is now output-aware: mean output fill is deducted from
mean input fill, mapped as 0.5·(1 + in - out). Input fill alone asks only
"how much work is waiting for me"; a node whose outputs are already full
cannot deliver, so running it just parks it again and wastes the slot
while the node that would drain that channel waits behind it. The
scheduler now favours whoever is furthest downstream of a bottleneck.

Also adds a network-level error listener. A node's exception was discarded
at the node boundary and survived only as a Closed event, which reports
that a node stopped but not why — that missing detail is what made the
above slow to diagnose. INode::set_network_error_callback plus
StaticNetwork::set_error_handler forward it to the application.

Tests: 121/121. test_backpressure_deadlock drives a five-node chain with
capacity-2 channels against a slow sink and fails on the old code. The
four test_pool_node overflow tests now assert parking rather than the
removed drop-and-report behaviour.

Known-incomplete: a rare hang remains, roughly 1 run in 20 against a 300s
timeout, down from every run failing. Committed because the fix is a large
strict improvement and the residual case needs its own reproduction.
2026-07-31 22:40:07 +02:00

443 lines
19 KiB
C++

#pragma once
#include "channel.hpp"
#include "diagnostics.hpp"
#include "fanout.hpp"
#include "inode.hpp"
#include "port.hpp"
#include "tmp/fanout_groups.hpp"
#include "tmp/topo_sort.hpp"
#ifdef KPN_WEB_DEBUG
#include "web_debug.hpp"
#include <memory>
#endif
#include <iostream>
#include <map>
#include <string>
#include <thread>
#include <tuple>
#include <type_traits>
#include <utility>
#include <vector>
namespace kpn {
// ── Edge descriptor ───────────────────────────────────────────────────────────
//
// Carries compile-time type info and runtime references to the two endpoints.
// Constructed by the edge() factory; consumed by make_network().
template<typename SrcNode, std::size_t SrcIdx,
typename DstNode, std::size_t DstIdx>
struct Edge {
using src_node_t = SrcNode;
using dst_node_t = DstNode;
static constexpr std::size_t src_idx = SrcIdx;
static constexpr std::size_t dst_idx = DstIdx;
SrcNode& src;
DstNode& dst;
};
template<typename SrcNode, std::size_t SrcIdx,
typename DstNode, std::size_t DstIdx>
auto edge(OutputPort<SrcNode, SrcIdx>, InputPort<DstNode, DstIdx> in)
-> Edge<SrcNode, SrcIdx, DstNode, DstIdx>; // deduction only; defined below
template<typename SrcNode, std::size_t SrcIdx,
typename DstNode, std::size_t DstIdx>
Edge<SrcNode, SrcIdx, DstNode, DstIdx>
edge(OutputPort<SrcNode, SrcIdx> out, InputPort<DstNode, DstIdx> in) {
return {out.node, in.node};
}
// ── StaticNetwork ─────────────────────────────────────────────────────────────
// node_label<NodeT>: returns Label NTTP as string_view if present, else empty.
template<typename NodeT, typename = void>
struct node_label_helper {
static constexpr std::string_view value = "";
};
template<typename NodeT>
struct node_label_helper<NodeT, std::void_t<decltype(NodeT::label())>> {
static constexpr std::string_view value = NodeT::label();
};
template<typename NodeT>
inline constexpr std::string_view node_label_v = node_label_helper<NodeT>::value;
// node_display_name<NodeT, UniqueTag>: Label if non-empty, else "node[UniqueTag]"
// Returned as std::string at runtime (called once at StaticNetwork construction).
template<typename NodeT>
std::string node_display_name() {
constexpr std::string_view lbl = node_label_v<NodeT>;
if constexpr (!lbl.empty()) {
return std::string(lbl);
} else if constexpr (requires { NodeT::is_fanout_node; NodeT::unique_tag; }) {
return "fanout[" + std::to_string(NodeT::unique_tag) + "]";
} else if constexpr (requires { NodeT::is_router_node; NodeT::unique_tag; }) {
return "router[" + std::to_string(NodeT::unique_tag) + "]";
} else if constexpr (requires { NodeT::is_filter_node; NodeT::unique_tag; }) {
return "filter[" + std::to_string(NodeT::unique_tag) + "]";
} else if constexpr (requires { NodeT::unique_tag; }) {
return "node[" + std::to_string(NodeT::unique_tag) + "]";
} else {
return "node[?]";
}
}
template<typename FanoutStorage, typename TopoNodeList>
class StaticNetwork : public INode {
public:
// FanoutStorage = std::tuple<FanoutNode<T0,N0>, ...> (owned, heap-allocated)
// TopoNodeList = tmp::TypeList<NodeA, NodeB, ...> sources-first
StaticNetwork(std::unique_ptr<FanoutStorage> fanouts,
std::vector<INode*> user_nodes_topo,
std::vector<INode*> fanout_ptrs,
std::vector<std::string> user_node_names,
std::vector<std::string> fanout_node_names,
std::vector<std::unique_ptr<IChannelProbe>> channel_probes)
: fanouts_(std::move(fanouts))
, user_nodes_topo_(std::move(user_nodes_topo))
, fanout_nodes_ptr_(std::move(fanout_ptrs))
, user_node_names_(std::move(user_node_names))
, fanout_node_names_(std::move(fanout_node_names))
, channel_probes_(std::move(channel_probes))
{}
~StaticNetwork() override { stop(); }
void start() override {
stop_flag_ = false;
start_time_ = clock_t::now();
if (event_handler_) {
for (std::size_t i = 0; i < user_nodes_topo_.size(); ++i) {
auto* node = user_nodes_topo_[i];
const auto& n = user_node_names_[i];
node->set_network_overflow_callback(
[this, n](auto ts) { event_handler_(n, NodeEvent::Overflow, ts); });
node->set_network_closed_callback(
[this, n](auto ts) { event_handler_(n, NodeEvent::Closed, ts); });
}
}
if (error_handler_) {
for (auto* node : user_nodes_topo_)
node->set_network_error_callback(error_handler_);
}
for (auto* n : user_nodes_topo_) n->start();
for (auto* n : fanout_nodes_ptr_) n->start();
#ifdef KPN_WEB_DEBUG
if (web_server_enabled_) {
web_server_ = std::make_unique<web_debug::WebDebugServer>(
web_debug_port_,
[this]() {
auto s = collect_snapshots();
return web_debug::to_json(s.nodes, s.channels, s.resources, s.elapsed_s, s.pools);
});
web_server_->start();
std::cerr << "[kpn] web debug UI: http://localhost:" << web_debug_port_ << "\n";
}
#endif
}
void stop() override { halt(); }
void halt() override {
stop_flag_ = true;
#ifdef KPN_WEB_DEBUG
if (web_server_) web_server_->stop();
#endif
for (auto it = fanout_nodes_ptr_.rbegin(); it != fanout_nodes_ptr_.rend(); ++it)
(*it)->stop();
for (auto it = user_nodes_topo_.rbegin(); it != user_nodes_topo_.rend(); ++it)
(*it)->stop();
}
// shutdown(): graceful drain in topological order (sources first).
// Stops source nodes, polls channels until empty, then stops each downstream layer.
void shutdown() override {
stop_flag_ = true;
#ifdef KPN_WEB_DEBUG
if (web_server_) web_server_->stop();
#endif
// user_nodes_topo_ is already in sources-first order.
// Stop each node and drain its output channels before moving on.
for (auto* n : user_nodes_topo_) {
n->stop();
drain_all_channels();
}
for (auto* n : fanout_nodes_ptr_) n->stop();
}
bool running() const override { return !stop_flag_; }
void set_name(std::string name) override { name_ = std::move(name); }
const NodeStats& stats() const override { static NodeStats dummy; return dummy; }
NodeSnapshot node_snapshot(const std::string& n, double) const override {
return {n, 0, 0, 0, 0, 0, 0, 0};
}
using EventHandler =
std::function<void(std::string_view node_name, NodeEvent,
std::chrono::steady_clock::time_point)>;
void set_event_handler(EventHandler h) { event_handler_ = std::move(h); }
/// Application-level error listener. Receives the exception any node's
/// function throws, after that node's own handler (if any) declined it.
/// Return true to skip the failed invocation and keep the node running,
/// false to let it stop. Without a listener the exception is discarded
/// and only a Closed event survives, which reports that a node stopped
/// but not why.
void set_error_handler(NodeErrorHandler h) { error_handler_ = std::move(h); }
#ifdef KPN_WEB_DEBUG
void set_web_debug_port(uint16_t port) { web_debug_port_ = port; }
// Called by DebugHub::register_network() so the hub owns the debug server.
void disable_web_server() { web_server_enabled_ = false; }
#endif
// Returns a snapshot of this network's nodes and channels for the DebugHub.
NetworkSnapshot network_snapshot() const {
auto s = collect_snapshots();
return {"", std::move(s.nodes), std::move(s.channels), s.elapsed_s};
}
// Register a shared resource so it appears in diagnostics and the debug UI.
// The probe must outlive this network (typically the resource is on the same stack).
void register_resource(const std::string& name, IResourceProbe* probe) {
resource_probes_.emplace_back(name, probe);
}
// Register a thread pool so it appears in diagnostics and the debug UI.
// The probe must outlive this network (typically the pool is on the same stack/shared_ptr).
void register_pool(const std::string& name, IPoolProbe* probe) {
pool_probes_.emplace_back(name, probe);
}
// Print diagnostics using compile-time node labels
void print_diagnostics(std::ostream& os = std::cerr) const {
os << "\n┌─ KPN++ StaticNetwork diagnostics ─────────────────────────────\n";
for (std::size_t i = 0; i < user_nodes_topo_.size(); ++i) {
auto snap = user_nodes_topo_[i]->node_snapshot(user_node_names_[i], 0.0);
os << "│ " << snap.name
<< " frames=" << snap.frames_processed
<< " ema=" << snap.ema_exec_ms << "ms\n";
}
os << "└────────────────────────────────────────────────────────────────\n";
}
FanoutStorage& fanouts_storage() { return *fanouts_; }
private:
struct Snapshots {
std::vector<NodeSnapshot> nodes;
std::vector<ChannelSnapshot> channels;
std::vector<ResourceSnapshot> resources;
std::vector<PoolSnapshot> pools;
double elapsed_s;
};
Snapshots collect_snapshots() const {
double elapsed_s = std::chrono::duration<double>(
clock_t::now() - start_time_).count();
std::vector<NodeSnapshot> nodes;
for (std::size_t i = 0; i < user_nodes_topo_.size(); ++i)
nodes.push_back(user_nodes_topo_[i]->node_snapshot(user_node_names_[i], elapsed_s));
for (std::size_t i = 0; i < fanout_nodes_ptr_.size(); ++i)
nodes.push_back(fanout_nodes_ptr_[i]->node_snapshot(fanout_node_names_[i], elapsed_s));
std::vector<ChannelSnapshot> channels;
for (auto& probe : channel_probes_)
channels.push_back(probe->snapshot());
std::vector<ResourceSnapshot> resources;
for (auto& [name, probe] : resource_probes_)
resources.push_back(probe->snapshot(name));
std::vector<PoolSnapshot> pools;
for (auto& [name, probe] : pool_probes_)
pools.push_back(probe->snapshot(name));
return {std::move(nodes), std::move(channels), std::move(resources), std::move(pools), elapsed_s};
}
void drain_all_channels() const {
bool any_full = true;
while (any_full) {
any_full = false;
for (auto& probe : channel_probes_) {
if (probe->snapshot().current_fill > 0) { any_full = true; break; }
}
if (any_full)
std::this_thread::sleep_for(std::chrono::milliseconds(1));
}
}
std::string name_;
bool stop_flag_{false};
std::unique_ptr<FanoutStorage> fanouts_;
std::vector<INode*> user_nodes_topo_;
std::vector<INode*> fanout_nodes_ptr_;
std::vector<std::string> user_node_names_;
std::vector<std::string> fanout_node_names_;
std::vector<std::unique_ptr<IChannelProbe>> channel_probes_;
std::vector<std::pair<std::string, IResourceProbe*>> resource_probes_;
std::vector<std::pair<std::string, IPoolProbe*>> pool_probes_;
EventHandler event_handler_;
NodeErrorHandler error_handler_;
clock_t::time_point start_time_;
#ifdef KPN_WEB_DEBUG
uint16_t web_debug_port_{9090};
bool web_server_enabled_{true};
std::unique_ptr<web_debug::WebDebugServer> web_server_;
#endif
};
// ── make_network ──────────────────────────────────────────────────────────────
template<typename... Edges>
auto make_network(Edges&&... edges) {
// 1. Expand edges — detect fan-outs, splice FanoutNodes
using FanoutSto = tmp::fanout_storage_t<std::decay_t<Edges>...>;
using ExpandedEdges = tmp::expanded_edges_t<std::decay_t<Edges>...>;
// 2. Duplicate-tag check — fires before cycle check for a cleaner error message
using UserNodes = typename tmp::all_node_types<tmp::TypeList<std::decay_t<Edges>...>>::type;
static_assert(!tmp::has_duplicate_tags_v<std::decay_t<Edges>...>,
"make_network: two nodes have the same (Func, UniqueTag) — they are "
"indistinguishable as graph vertices. Add a UniqueTag: "
"make_node<func, \"label\", 1>(capacity)");
// 3. Cycle check
using Topo = tmp::topo_sort<ExpandedEdges>;
constexpr bool has_cycle = Topo::has_cycle;
static_assert(!has_cycle,
"make_network: graph contains a directed cycle");
// 4. Construct owned fanout storage on the heap (FanoutNode has jthread — not moveable)
auto fanout_storage = std::make_unique<FanoutSto>();
// 5. Collect unique user node pointers + their display names, in edge-declaration order
std::vector<INode*> user_node_ptrs;
std::vector<std::string> user_node_names;
auto collect = [&](auto& e) {
using SrcT = std::decay_t<decltype(e.src)>;
using DstT = std::decay_t<decltype(e.dst)>;
auto* s = static_cast<INode*>(&e.src);
auto* d = static_cast<INode*>(&e.dst);
if (std::find(user_node_ptrs.begin(), user_node_ptrs.end(), s) == user_node_ptrs.end()) {
auto sname = node_display_name<SrcT>();
user_node_ptrs.push_back(s);
user_node_names.push_back(sname);
s->set_name(sname);
}
if (std::find(user_node_ptrs.begin(), user_node_ptrs.end(), d) == user_node_ptrs.end()) {
auto dname = node_display_name<DstT>();
user_node_ptrs.push_back(d);
user_node_names.push_back(dname);
d->set_name(dname);
}
};
(collect(edges), ...);
// 5. Wire all expanded SimpleEdges.
// find_node<NodeT>: searches fanout storage then user edge pack, returns NodeT*.
// Uses if constexpr in a fold so mismatched types never reach assignment.
auto find_node = [&]<typename NodeT>() -> NodeT* {
NodeT* ptr = nullptr;
std::apply([&](auto&... fn) {
([&](auto& node) {
if constexpr (std::is_same_v<std::decay_t<decltype(node)>, NodeT>)
if (!ptr) ptr = &node;
}(fn), ...);
}, *fanout_storage);
if (!ptr) {
([&](auto& e) {
if (!ptr) {
if constexpr (std::is_same_v<std::decay_t<decltype(e.src)>, NodeT>)
ptr = &e.src;
else if constexpr (std::is_same_v<std::decay_t<decltype(e.dst)>, NodeT>)
ptr = &e.dst;
}
}(edges), ...);
}
return ptr;
};
// Pre-pass: build fanout_id → source display name map so fanout nodes
// can be named after the node feeding them (e.g. "capture_fanout").
std::map<std::size_t, std::string> fanout_src_name;
[&]<typename... SEs>(tmp::TypeList<SEs...>) {
([&]<typename SE>(SE) {
using DstNode = typename SE::dst_node_t;
using SrcNode = typename SE::src_node_t;
if constexpr (requires { DstNode::is_fanout_node; })
fanout_src_name.emplace(DstNode::unique_tag, node_display_name<SrcNode>());
}(SEs{}), ...);
}(ExpandedEdges{});
// Helper: display name for any node type, resolving fanouts to "src_fanout".
auto node_name = [&]<typename NodeT>() -> std::string {
if constexpr (requires { NodeT::is_fanout_node; }) {
auto it = fanout_src_name.find(NodeT::unique_tag);
return it != fanout_src_name.end() ? it->second + "_fanout"
: node_display_name<NodeT>();
} else {
return node_display_name<NodeT>();
}
};
std::vector<std::unique_ptr<IChannelProbe>> channel_probes;
auto wire_one = [&]<typename SE>(SE) {
using SrcNode = typename SE::src_node_t;
using DstNode = typename SE::dst_node_t;
using out_t = std::tuple_element_t<SE::src_idx, typename SrcNode::return_tuple>;
constexpr std::size_t SrcIdx = SE::src_idx;
constexpr std::size_t DstIdx = SE::dst_idx;
auto* src = find_node.template operator()<SrcNode>();
auto* dst = find_node.template operator()<DstNode>();
if (src && dst) {
auto& ch = dst->template input_channel<DstIdx>();
src->template set_output_channel<SrcIdx>(&ch);
std::string ch_name = node_name.template operator()<SrcNode>() + ":" + std::to_string(SrcIdx)
+ " \xe2\x86\x92 " // UTF-8 →
+ node_name.template operator()<DstNode>() + ":" + std::to_string(DstIdx);
channel_probes.push_back(std::make_unique<ChannelProbe<out_t>>(ch, ch_name));
}
};
[&]<typename... SEs>(tmp::TypeList<SEs...>) {
(wire_one(SEs{}), ...);
}(ExpandedEdges{});
// 6. Collect fanout node pointers and names
std::vector<INode*> fanout_ptrs;
std::vector<std::string> fanout_node_names;
std::apply([&](auto&... fn) {
([&](auto& node) {
using NodeT = std::decay_t<decltype(node)>;
auto fname = node_name.template operator()<NodeT>();
static_cast<INode&>(node).set_name(fname);
fanout_ptrs.push_back(static_cast<INode*>(&node));
fanout_node_names.push_back(fname);
}(fn), ...);
}, *fanout_storage);
// 7. Construct and return the StaticNetwork
using Net = StaticNetwork<FanoutSto, typename Topo::topo>;
return Net(std::move(fanout_storage),
std::move(user_node_ptrs),
std::move(fanout_ptrs),
std::move(user_node_names),
std::move(fanout_node_names),
std::move(channel_probes));
}
} // namespace kpn