Live 4-process E2E run (3-node controller cluster + xcecd_virtual worker + xce_scheduler + xce_apiserver, all on the full 3-candidate list) with the primary killed mid-session. Worker reconnect is solidly proven with real log evidence and unchanged PIDs across all three components throughout. Scheduler's rejoin path was never naturally exercised (no job reached its picker/allocator) -- same partial-coverage shape already accepted for task 3.3. Surfaced a likely-real, structural gap in the already-approved task 4.3/4.4 apiserver rejoin mechanism (bug-039): XCECLI_FF_CALL can silently self-heal the transport within a single heartbeat tick, making the cross-tick failure->success transition invisible to heartbeat.go's detector on any failover fast enough to complete within one heartbeat interval -- which is most real single-election failovers in this codebase. Also found bug-040 (job pushes never reach apiserver's store), very likely a downstream symptom of bug-039's rejoin gap via on_apiserver_join()'s per-process, unreplicated apiserver registration list. Both logged with detailed, independently-verified root-cause analysis for a dedicated follow-up -- fixing them is outside this task's cross-component-validation boundary. No src/ changes -- live validation only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NNB4zGPCsHyDUCDJyMbiQN |
||
|---|---|---|
| .. | ||
| settings | ||
| specs | ||
| steering | ||