test_live_config_push_against_standby_returns_standby_rejection_not_redirect
locked in the OLD controller behavior (a standby flatly rejects
OP_upgradeConfig with no redirect fields). That gap is now closed:
myreq_handler's gate-rejection path calls fill_redirect_reply() on
every -XCE_ERR_STANDBY rejection (bug-1700), and handle_ctrlhello() now
actually populates primary_host/primary_port from the static --peers
list (bug-1701) -- a standby pushing config now redirects to the
primary and the push succeeds. Renamed to
test_live_config_push_against_standby_redirects_and_succeeds and
rewritten to assert the new behavior.
_assert_primary_committed() changed from a single-shot log read to a
poll loop (up to 5s) for the "config activated from committed
marker..." line -- accounts for the documented async gap between the
majority-ack-gated client reply and the primary's own local WAL-apply-
pump activation, a real race the old single-shot read could hit.
meson-build.sh: wire src/tools and each build profile's controller
subdir into the isolated repo-mirror the E2E tests run against, and
export XCEADMIN_BIN explicitly, so these live tests actually find real
binaries instead of always skipping.
Verified: xceadmin suite 288/288 (incl. both tests in this file);
controller suite 1220/1222 (only the pre-existing bug-1561 flake).
Updated admin-config-tool design.md and roadmap.md to mark the
dispatch-gate and redirect-wiring gaps RESOLVED, matching this test's
new assertions and the live evidence.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QxaSzHRB7pmYFp4cVnwkA
/kiro-discovery investigation of removing src/frontend/Makefile in favor
of Meson as the sole build system. Records the quick investigation's
findings (5 boundary seams, current-state evidence that build-cli/
build-server are already dead from /xce-build's perspective and
build-server is already broken today) and the user's confirmed decision
to fully remove the Makefile rather than keep a compat wrapper.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AgYRU3zDTqNGV5Jk8aiKLg
Full spec scaffold (requirements/design/research/tasks all approved,
ready_for_implementation=true) plus a roadmap backlog item recorded from
the dirty-content-fingerprint spec's task 4.3 findings (Makefile/meson
fingerprint-computation duplication risk).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Surfaced while investigating a user request to show total/current values
for GLOBAL/SHARED resources in the scheduler REPL's "info resource"
command -- researched and confirmed that concept doesn't exist anywhere
in this codebase (resourcemap config field dropped at build time, capnp
fields commented out, worker.c actively rejects XCE_RESF_SHARED resources
today). This spec already owns closing that gap; it just never made it
into the roadmap's backlog list.
- xce_test shell command support: was already implemented (xce-test-e2e-
primitives task 2.1, closed 2026-06-19) -- stale "not started" entry
corrected.
- scheduler-core tasks 2.1-2.6 _Verified: backfill: was already done
(commit d7b49690, 2026-08-08) -- corrects a status line I myself wrote
incorrectly earlier today by trusting the original 2026-08-07 finding's
wording without re-checking the file.
- job_ctrl direct-mutate-path investigation: completed. Real risk
confirmed (pre-commit state pushed live to scheduler + apiserver/clients
with no rollback on commit failure), scope wider than originally framed
(on_job_stop/on_job_resume/on_job_ckpt share the same pattern, fed by
ordinary traffic not just rare admin commands), plus a separate
read-lock-during-mutation bug found along the way. Not promoted to a
spec yet -- scope/framing corrected first per this session's decision.
- OP_upgradeConfig standby redirect verification: investigated. The
item's own premise doesn't match production (a standby never actually
returns a redirect for this opcode -- a documented, deliberate feature
gap, not a missing test); coverage confirmed already complete on both
the client mock side and a live 3-node regression test.
design.md's own "Open Questions/Risks" list (items 5 and 6) still described
the OPCode_schedAdmin admission gap and the tlog activation gap as open/
deferred, even though both were fixed earlier in this session -- marked
FIXED with pointers to the actual change. research.md's schedAdmin
decision record and its Risks & Mitigations bullet got the same treatment
(SUPERSEDES note, historical record preserved rather than rewritten).
tasks.md's 2026-08-07 GO/NO-GO follow-up list gained a status-update line:
follow-up 1 (tlog) is now fixed, follow-up 2 (fcfs.c/builtin_fcfs.c
traceability) is only partially closed (design.md text was already
correct; tasks 2.1-2.6 still lack _Verified: lines -- flagged, not
fixed here since I have no fresh verification evidence for those tasks).
roadmap.md's scheduler-core summary line corrected to attribute each gap
to the right source (2026-08-07 validate-impl vs today's separate
design-review pass) instead of conflating them.
Every early-exit branch in the worker_list scan either re-tested the same
node forever (bare `continue`) or returned out of the whole function on the
first DONE/EXITED worker, so job array elements past the head of the list
never got their on_job_start report. Capture `next` before touching each
node and always advance to it. Affects xcecd_virtual and real xcecd alike
(same DBUS_MANAGER-gated function).
Also records a roadmap follow-up (xcecd_virtual concurrent-job support,
MXJ > 1) discovered while investigating this bug.
- config-upgrade-snapshot, raft-adapter-conformance: correct stale
phase metadata (both showed "tasks-generated" though every task was
already complete) and record their validate-impl GO decisions with
known-limitations notes (bug-1471/bug-1472-remainder, bug-1474
angle-b) for future spec ownership.
- component-failover-adoption: catch up design/requirements/research
docs to the Phase 6 fix (bug-1548/bug-1549, commit d479b707) that
landed after this spec was already marked closed.
- Promote 3 roadmap backlog items to specs (2026-08-10 discovery):
component-rejoin-resync (apiserver/scheduler rejoin resync),
worker-rejoin-resync (worker job-fact resync, hard prereq for
raft-apply-decoupling's facts half), scheduler-sim-diagnostics
(open-ended scheduler REPL debug/troubleshoot capability).
- roadmap.md updated to reflect all of the above.
- kiro-steering/kiro-steering-custom SKILL.md: fix stale template path
(templates/steering/ -> .kiro/settings/templates/steering/) plus
minor doc corrections in the other kiro-spec-* skills.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
These edits (three-thread picker correction in brief.md, record_dir
path-validation and kvmgr-audit requirements in requirements.md,
OPCode_schedAdmin folded into the peer-authentication-hardening
backlog item in roadmap.md) were made during earlier requirements
review and design work but left uncommitted. Reviewed for accuracy
against the completed implementation — all still correct, persisting
as-is.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- ha-async-io.md: document the same-batch endpoint lifetime hazard
(bug-1447) and its ref-whole-batch-before-dispatch fix pattern
- xce-audit-invariants SKILL.md: add object-lifetime-within-a-dispatch-batch
as a 6th audit dimension, so a future pass checks other batch loops
(vtaskmgr drain, peer-list iteration) for the same shape
- product.md: replace the stale "HA consistency family is the current
focus" framing with the actual 2026-07-18 status (umbrella complete,
9-spec validate-impl batch closed, job-array-single-wal-record shipped)
- roadmap.md: correct 5 entries that were stale relative to actual spec
status (ha-consistency-invariants/ha-wal-replication/libxce-failover
pulled out of "In Flight", ha-term-safety and raft-drive-async marked
implemented+GO, raft-sim-conformance and job-array-single-wal-record
given closure annotations) -- struck through, not deleted, per this
file's own additive-update convention
- testing.md: document that meson's aggregate Fail:0 can mask Unity
sub-group failures -- always check the standalone binary's own summary
- src/controller/CLAUDE.md: fix a stale line-citation for the votedFor
atomicity invariant (had drifted onto unrelated post-election code)
raft-apply-decoupling: 32/32 tasks complete, validate-impl GO (79/79 test
groups, live smoke boot verified) -- update roadmap's In Flight entry with
outcome, residuals, and follow-ups.
raft-drive-async: all 17 sub-tasks were already [x] but the 4 phase-header
checkboxes were never ticked, leaving tasks.md looking incomplete. Confirmed
via source (g_drive_async_configpush_offload defaults to 1) and prior
session notes that all 4 phases genuinely landed and flipped ON in
production; checkboxes now match reality.
Adds the Raft Apply-Ordering & Hook Decoupling roadmap phase: canonical
pre-hooks -> raft commit -> apply -> notify ordering, with the raft protocol
step lifted out of the notification hook fan-out (root of the bug-795/828/
851/1060 divergence family). Decision: apply-after-commit via a single
shared apply_committed_entry() seam reused by online apply and restart
replay.
- raft-drive-async: full spec docs (brief/requirements/research/design)
committed alongside its already-tracked tasks.md; async-drive prerequisite
for the apply-decoupling work, sequenced to start first (threatens
failover liveness today).
- raft-apply-decoupling: new spec (brief + facts-in-raft.md) covering the
apply seam and the facts (WAL_MODE_RETAIN) apply-timing question (open
decision D1).
- job-array-single-wal-record: promoted from backlog to a tracked spec,
sequenced after raft-apply-decoupling.
- ha-consistency-invariants: add criterion 2.9 (Facts-Apply invariant),
documenting the current on-receipt fact-reaction deviation this family
will reconcile.
- raft-commit-safety / raft-log-replication: reconcile spec.json phase to
"implemented" (both were mislabeled tasks-generated despite full 18/18
and 24/24 task completion + raft_sim GO).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SDWVNQLKxqTVHh42cH7rV3
Full controller suite green under ASAN (508 PASS / 0 FAIL / 0 sanitizer
aborts, independently re-censused). All 16 tasks complete; barrier
before reply proven across all restructured opcodes; classification
guard active; barrier-before-state-removal precondition for
ha-single-leader-role satisfied. spec phase=implemented; roadmap
updated. Sub-spec 2 of 4 of the ha-consistency-invariants umbrella.
