Commit graph

6 commits

Author SHA1 Message Date
lei.chen
0c8607345e merge: sync Rongsong's scheduler-framework (c5b172c7..3442a57d, 781 commits)
Merge origin/scheduler-framework into merge-sf-20260903 (97 commits on
ours: bug-1490/1494/1498/1509/1513/1521/1522 fixes, PEND-reason P1,
scheduler-agent Phase 2, bug-924 owner-pid shim).

Bookkeeping excluded per policy: .kiro/, .wolf/, CHANGES.md, reports/
(92 files dropped); .claude/ + CLAUDE.md keep ours.

Key theirs-side changes absorbed:
- bug-1938/1960: QUEUE_STAT_CLOSED no longer gates dispatch (closed
  queue refuses new submissions only; pending jobs still run) --
  supersedes our QUEUE_CLOSED preason on the dispatch path;
  sched_queue_status_reason drops that branch
- bug-1846: config push now stashes and applies between scheduling
  rounds (sched_stash_config/sched_apply_pending_config), applied at
  the same main-thread round boundary that preserves bug-1513
  serialization
- bug-1760: sched_check_pending_rejoin exposed +
  xcecli_is_connection_broken public predicate
- bug-1936: execuser/exechome/execcwd/exdata carried on submission
  (superset of our bug-1500 execuser fallback; our job-typed
  resolveExecUser removed as dead code)
- bug-1945~1962: apiserver auth hardening (HS256 pinning, default-
  secret rejection, logout revocation, bcrypt, RBAC) -- our API-key
  branch folded into controllers.ParseTokenClaims so both coexist
- bug-1736: rawRunUsage job run-usage fields in the Go model
- xce_test relocated src/controller/testsuite/test.c ->
  src/harness/test.c; controller suite split into
  controller_test.c + fixtures; their clitest fixtures spawn xce_test
  mid-suite under `meson test -j`

Ours preserved across the merge:
- PEND-reason P1 (codes 1-9, preason.h, job_preason machinery) --
  gate re-ported into their relocated allocator (sched_election.c)
  after plugin_job_election, with attach on the allocator thread
  only (bug-1521 race discipline)
- PEND-reason P0 main_reason semantics RECONCILED to theirs: the
  merge initially re-carried our first-detail-add-promotes-main_reason
  fallback in add_job_preason (4e2ac86), but their whole trace/verify
  stack treats "detail-only => main_reason 0" as a contract --
  sched_trace_capture counts main_reason as its own occurrence
  ("Expected 2 Was 3" in test_trace_capture), the outcome renderer
  prints "main reason: none" for detail-only PENDs (test_repl_trace_e2e),
  and the committed .replay/.sim_verify baselines encode main_reason=0
  (test_repl_trace_baseline byte compare, test_trace_equivalence
  off-vs-traced). The fallback is dropped from the shared lib;
  the user-visible "overall reason" survives as an API-layer view:
  rawJobToModel (convert_pure.go) promotes the first detail row to
  main_reason_name when main_reason is 0 (main_reason VALUE marshals
  unchanged). add_job_preason keeps only the NULL-reasons guard
- bug-1498: sched_job_is_ready rejects non-PEND/suspended jobs
- bug-1490/1781: xce_jobctrl_t.action as capnp union discriminant
  (their jobCtrlUnionDiscriminant, same semantics)
- xcecli typed PendingReason field (53c00834), bug-1522 reconnect
  resend, xcecli_force_reconnect + their is_connection_broken both
  exported in client.h/libclient
- t1/t2/t3 test-node hostconfig in src/configs/hosts.yaml (bug-1494),
  /xce/v1/events + /xce/v1/apikeys whitelist routes (agent Phase 2)
- lib ASAN_OPTIONS=detect_leaks=0 test env + their oom-test block
- drop-audit: our bug-1504 fix covered by their bug-1680 (no action)

bug-924 follow-up (replaces the 67f3f6ad scratch-dir wipe): the
take-theirs audit's one "drop" was re-evaluated and REVERTED instead of
re-applied -- a global `find /tmp -name 'xce_*' -exec rm -rf` at suite
start/exit races the LIVE /tmp/xce_* fixtures of sibling suites under
`meson test -j` (harness_unit_test's clitest fixtures spawn xce_test as
a child mid-suite; the wipe destroyed the parent's fixtures ->
test_test_cli failures). Coverage instead:
- mkdtemp_clean shim gains mkdtemp_clean_mark/sweep_since
  (testsuite_support/mkdtemp_clean.[ch])
- new case_scratch_sweep.[ch]: aggregate runners (controller_test.c,
  harness_test.c -- hundreds of cases per process) call
  case_scratch_sweep_begin/_end_ok from setUp/tearDown; a PASSED case's
  dirs are swept immediately in tearDown (after engine reaping), FAILED
  cases keep their dirs for post-mortem; XCE_MKDTEMP_KEEP_ALL=1
  disables. One full controller_unit_test run was observed to leak 12G+
  to ENOSPC without this, cascading into unrelated "Expected Non-NULL"
  failures
- harness_unit_test_deps links the shim library as well

Test reconciliations on adopt:
- core-ready-gates: sched_queue_is_ready now gates only
  INACT_ADMIN|INACT_WINDOW (bug-1938/1960); CLOSED queues elect --
  covered end-to-end by test_queue_admin_control.c
- test_worker_resync: the 4 restart-replay RUN assertions predate the
  bug-1509 B3 marking (8858afed) landing on the same lineage; replayed
  RUN jobs now assert RUN|UNKWN|TRANSIENT (TWR_ASSERT_REPLAYED_RUN);
  terminal DONE/EXIT assertions unchanged
- convert_pure.go: our job-typed resolveExecUser deleted (superseded
  by their bug-1780/1936 two-arg version)
- test_preason: the 3 P0-promotion assertions reverted to the
  detail-keeps-main-0 contract; new tests cover limit-code detail
  rows and explicit-main-wins-over-later-details; apiserver
  convert_test.go gains the detail-only name-promotion cases

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-14 05:17:03 +00:00
f5fbc6bcac
feat(eliminate-frontend-makefile): task 3.3 - update live documentation (11 files)
Removes every live current-fact reference to src/frontend/Makefile
targets or Makefile-specific fixed paths across 11 docs (steering,
frontend/CLAUDE.md, root CLAUDE.md, deployment docs, and two skill
docs), replacing with the Meson-based equivalents established by
tasks 1-3.2: /xce-build apiserver, meson test --suite apiserver,
direct tests/regression/run.sh invocation.

Round-1 review rejected the first pass: xce-deploy-ha/SKILL.md had
over-corrected by deleting all mention of deploy-ha.sh's still-real
/tmp/apiserver legacy fallback (that script's discovery logic is
unchanged, unlike run-resource-verify.sh's, which genuinely lost its
fallback in task 3.1 -- xce-test-resources/SKILL.md's identical-
looking edit was correct for that reason). Fixed by restoring an
accurate, Makefile-free description of the real fallback at all four
touched sites. Re-reviewed and approved.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XYdtrEWWZfKRRGED7raEYg
2026-09-03 14:22:54 +00:00
lei.chen
9fe14c1512 docs(scheduler-agent): update design doc with cluster verification results
- Design doc v1.2: Phase 0-1 cluster verification status (read + submit OK,
  control HTTP 500 open bug-1490), bug table for whitelist/job_status fixes,
  mcp_operator queue:control note.
- Root + frontend CLAUDE.md: reflect verified state and apiserver bugs found.
- wolf memory/anatomy/buglog: log deployment session, bug-1489/bug-1490.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-01 09:54:49 +00:00
Lei Chen
e4af0cac73 Phase 0: scheduleragent + mcp-server read-only agent framework
Complete the natural-language scheduler agent's Phase 0 read-only pipeline:

Task #2 (API Key auth mechanism):
- New store/apikey.go: APIKey store (hash, validate, expiry)
- controllers/apikey.go: GenerateAPIKey finally persists + registers RBAC role
- controllers/auth.go: GetUsernameFromToken recognizes xce_ak_ prefix
- middleware/validatehandler.go: isValidToken handles API Key
- config/rbac.yaml: mcp_readonly role (view-only permissions)
- 6 new test files covering unit + integration full-chain

Task #3 (schedulerclient Go library):
- types.go: URM types (URMJob/URMQueue/URMNode/JobStatus/Metrics)
- client.go: read-only Client interface (6 methods)
- httpclient.go: resty-based HTTP implementation
- statusmap.go + _test.go: TDD status mapping (33 tests)

Task #4 (mcp-server binary):
- cmd/main.go: entry, reads XCE_API_URL/XCE_API_KEY, registers 6 MCP tools
- internal/app.go: tool handlers wiring schedulerclient to MCP stdio
- Uses github.com/mark3labs/mcp-go@v0.57.0 for stdio transport

All: go vet clean, go build clean, schedulerclient 33/33 tests pass.
mcp-server binary ~11MB arm64, CGO_ENABLED=0.

Architecture:
  Claude Code --stdio--> mcp-server --HTTP--> apiserver REST API --cgo--> C backend

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-30 11:16:36 +08:00
06135954ff chore: baseline snapshot — prior-session work + ha-consistency-invariants spec family
Pre-existing uncommitted work: ha-repl-hardening payload data plane (7.x),
heartbeat liveness fix (Req 1.3), connection-reuse ASAN fixes, doc cleanup.
New: ha-consistency-invariants umbrella (requirements/design/research) and
its four implementation sub-specs (specs generated via batch, cross-spec
review applied), steering roadmap + product.md Raft+CP decision.
2026-06-11 13:21:50 +00:00
c9dc8c4676 Merge Go frontend (apiserver + xcecli) into meson build 2026-04-14 11:23:18 +08:00