Rongsong Shen
177e9b292a
Successor to scheduler-primary-lifecycle: a new per-node xce-node-manager component (one per host, role-assigned: manager/worker/apiserver/ reverse-proxy) owns starting/stopping XCE components for its node's role -- including starting/stopping the scheduler on a manager node in reaction to that node's controller reporting itself primary vs. standby over a local heartbeat (Unix domain socket, client logic shared via libxce) -- rather than the controller directly spawning/killing its own scheduler process. Carries forward scheduler-primary-lifecycle's underlying requirements (bug-1760's reactive-rejoin deadlock, wedge detection, graceful shutdown, controller-resilience invariants). Design review (kiro-validate-design) fixed 3 issues before GO: a uniform-single-OS-user assumption that was factually wrong given xce-deploy-ha's real worker daemon runs as root (resolved: root only on worker-role nodes); Requirement 4.3's wedge-detection mechanism verified directly against the actual scheduler code (sched_job_picker's cbuf_get() does an unbounded wait when idle, confirming the naive design would false-positive -- fixed by swapping to the existing cbuf_timedget() bounded-deadline variant); and PortCheckMonitor's TCP connect made explicitly non-blocking to match HeartbeatListener's own discipline. Tasks not yet generated. Checkpointing uncommitted spec work found in this shared worktree. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YAWMhfRJFnmmnG8h97yUDd
25 lines
4.3 KiB
JSON
25 lines
4.3 KiB
JSON
{
|
|
"feature_name": "xce-node-manager",
|
|
"created_at": "2026-09-03T03:45:42Z",
|
|
"updated_at": "2026-09-03T06:38:35Z",
|
|
"language": "en",
|
|
"phase": "design-generated",
|
|
"supersedes": "scheduler-primary-lifecycle",
|
|
"approvals": {
|
|
"requirements": {
|
|
"generated": true,
|
|
"approved": true,
|
|
"validation_note": "kiro-spec-requirements 2026-09-02: 8 requirement areas / 32 acceptance criteria drafted, grounded in in-conversation codebase investigation (ha.c primary_heartbeat_fn/ha->role, process_worker_manager.c's existing fork()-hazard precedent, process_manager.c/h supervision shape, controller_connector.c candidate-list pattern, hosts.yaml config format, xce-deploy-ha tooling, product.md/structure.md/tech.md/roadmap.md steering) plus the fully-read superseded scheduler-primary-lifecycle design.md. Requirements Review Gate run inline during drafting (not as a separate pass): mechanical checks (numeric IDs, EARS-format ACs, no implementation-language leakage) passed; two draft issues caught and fixed before writing -- Requirement 3's original job-durability AC used a non-EARS 'Regardless of...' opening (rewritten to a ubiquitous 'The Node Manager shall ensure...' form), and Requirement 7 originally included a non-AC scope-exclusion bullet (moved to Boundary Context's Out of scope list, replaced with a proper adoption-logging AC). Auto-approved 2026-09-03 via `/kiro-spec-design xce-node-manager -y`."
|
|
},
|
|
"design": {
|
|
"generated": true,
|
|
"approved": true,
|
|
"validation_note": "kiro-spec-design 2026-09-03: light-to-moderate discovery, see prior note for discovery findings and synthesis (build-vs-adopt rejection of a generic external supervisor; ComponentSupervisor/LivenessMonitor generalization; justified Cap'n Proto deviation). kiro-validate-design 2026-09-03: independent critical review (not the same pass as authoring) found 3 real issues, GO withheld pending fixes -- (1) uniform single-OS-user privilege assumption was factually wrong: xce-deploy-ha's own tooling documents the real (non-virtual) xcecd worker daemon runs as root, unaddressed by the original design; resolved per user decision -- xce-node-manager runs as root only on a node whose assigned role includes worker, ordinary service account otherwise, Security Considerations and research.md's Decision log rewritten accordingly. (2) Requirement 4.3's wedge-detection mechanism depended on the Scheduler's round loop ticking on a bounded cadence even when idle, left as an unverified 'confirm during implementation' open question despite research.md's own cited evidence (bug-1760's reactive/idle-silent root cause) pointing the other way; verified directly against the actual code via codegraph_explore -- sched_job_picker's cbuf_get() (src/lib/mylock.c:144) does an UNBOUNDED pthread_cond_wait when idle, confirming the naive design would cause false-positive wedge detection on every idle Scheduler. Fixed: swap to the already-existing cbuf_timedget() bounded-deadline variant with heartbeat calls on both the real-job and timeout paths; documented as a resolved design decision, not deferred. (3) PortCheckMonitor's TCP connect was never specified as non-blocking despite HeartbeatListener explicitly being so, and ComponentSupervisor's own text incorrectly claimed the Node Manager loop 'has no other liveness-critical responsibility to protect' -- both corrected: PortCheckMonitor's connect is now explicitly non-blocking (O_NONBLOCK + bounded poll/select), and the misleading framing was rewritten to state the loop IS liveness-critical for every role it supervises. Two further scope clarifications volunteered by the user during the fix cycle, applied directly: the heartbeat call site is explicitly the Scheduler's live driver (scheduler.c/sched_job_picker) only, never the pure core (core.c/core_events.c/core_session.c/core_clock.c, scheduler-core spec's own zero-controller-I/O invariant, now cited as a fourth cross-checked project invariant) or the simulation driver (sim.c, added as an explicit Non-Goal/Out-of-Boundary item -- simulation-mode Scheduler runs standalone, entirely outside xce-node-manager's awareness). All fixes applied directly to design.md/research.md in the same session; re-reviewed inline, no further issues found. GO."
|
|
},
|
|
"tasks": {
|
|
"generated": false,
|
|
"approved": false
|
|
}
|
|
},
|
|
"ready_for_implementation": false
|
|
}
|