merge: sync Rongsong's scheduler-framework (c5b172c7..3442a57d, 781 commits)
Merge origin/scheduler-framework into merge-sf-20260903 (97 commits on ours: bug-1490/1494/1498/1509/1513/1521/1522 fixes, PEND-reason P1, scheduler-agent Phase 2, bug-924 owner-pid shim). Bookkeeping excluded per policy: .kiro/, .wolf/, CHANGES.md, reports/ (92 files dropped); .claude/ + CLAUDE.md keep ours. Key theirs-side changes absorbed: - bug-1938/1960: QUEUE_STAT_CLOSED no longer gates dispatch (closed queue refuses new submissions only; pending jobs still run) -- supersedes our QUEUE_CLOSED preason on the dispatch path; sched_queue_status_reason drops that branch - bug-1846: config push now stashes and applies between scheduling rounds (sched_stash_config/sched_apply_pending_config), applied at the same main-thread round boundary that preserves bug-1513 serialization - bug-1760: sched_check_pending_rejoin exposed + xcecli_is_connection_broken public predicate - bug-1936: execuser/exechome/execcwd/exdata carried on submission (superset of our bug-1500 execuser fallback; our job-typed resolveExecUser removed as dead code) - bug-1945~1962: apiserver auth hardening (HS256 pinning, default- secret rejection, logout revocation, bcrypt, RBAC) -- our API-key branch folded into controllers.ParseTokenClaims so both coexist - bug-1736: rawRunUsage job run-usage fields in the Go model - xce_test relocated src/controller/testsuite/test.c -> src/harness/test.c; controller suite split into controller_test.c + fixtures; their clitest fixtures spawn xce_test mid-suite under `meson test -j` Ours preserved across the merge: - PEND-reason P1 (codes 1-9, preason.h, job_preason machinery) -- gate re-ported into their relocated allocator (sched_election.c) after plugin_job_election, with attach on the allocator thread only (bug-1521 race discipline) - PEND-reason P0 main_reason semantics RECONCILED to theirs: the merge initially re-carried our first-detail-add-promotes-main_reason fallback in add_job_preason (4e2ac86), but their whole trace/verify stack treats "detail-only => main_reason 0" as a contract -- sched_trace_capture counts main_reason as its own occurrence ("Expected 2 Was 3" in test_trace_capture), the outcome renderer prints "main reason: none" for detail-only PENDs (test_repl_trace_e2e), and the committed .replay/.sim_verify baselines encode main_reason=0 (test_repl_trace_baseline byte compare, test_trace_equivalence off-vs-traced). The fallback is dropped from the shared lib; the user-visible "overall reason" survives as an API-layer view: rawJobToModel (convert_pure.go) promotes the first detail row to main_reason_name when main_reason is 0 (main_reason VALUE marshals unchanged). add_job_preason keeps only the NULL-reasons guard - bug-1498: sched_job_is_ready rejects non-PEND/suspended jobs - bug-1490/1781: xce_jobctrl_t.action as capnp union discriminant (their jobCtrlUnionDiscriminant, same semantics) - xcecli typed PendingReason field (53c00834), bug-1522 reconnect resend, xcecli_force_reconnect + their is_connection_broken both exported in client.h/libclient - t1/t2/t3 test-node hostconfig in src/configs/hosts.yaml (bug-1494), /xce/v1/events + /xce/v1/apikeys whitelist routes (agent Phase 2) - lib ASAN_OPTIONS=detect_leaks=0 test env + their oom-test block - drop-audit: our bug-1504 fix covered by their bug-1680 (no action) bug-924 follow-up (replaces the67f3f6adscratch-dir wipe): the take-theirs audit's one "drop" was re-evaluated and REVERTED instead of re-applied -- a global `find /tmp -name 'xce_*' -exec rm -rf` at suite start/exit races the LIVE /tmp/xce_* fixtures of sibling suites under `meson test -j` (harness_unit_test's clitest fixtures spawn xce_test as a child mid-suite; the wipe destroyed the parent's fixtures -> test_test_cli failures). Coverage instead: - mkdtemp_clean shim gains mkdtemp_clean_mark/sweep_since (testsuite_support/mkdtemp_clean.[ch]) - new case_scratch_sweep.[ch]: aggregate runners (controller_test.c, harness_test.c -- hundreds of cases per process) call case_scratch_sweep_begin/_end_ok from setUp/tearDown; a PASSED case's dirs are swept immediately in tearDown (after engine reaping), FAILED cases keep their dirs for post-mortem; XCE_MKDTEMP_KEEP_ALL=1 disables. One full controller_unit_test run was observed to leak 12G+ to ENOSPC without this, cascading into unrelated "Expected Non-NULL" failures - harness_unit_test_deps links the shim library as well Test reconciliations on adopt: - core-ready-gates: sched_queue_is_ready now gates only INACT_ADMIN|INACT_WINDOW (bug-1938/1960); CLOSED queues elect -- covered end-to-end by test_queue_admin_control.c - test_worker_resync: the 4 restart-replay RUN assertions predate the bug-1509 B3 marking (8858afed) landing on the same lineage; replayed RUN jobs now assert RUN|UNKWN|TRANSIENT (TWR_ASSERT_REPLAYED_RUN); terminal DONE/EXIT assertions unchanged - convert_pure.go: our job-typed resolveExecUser deleted (superseded by their bug-1780/1936 two-arg version) - test_preason: the 3 P0-promotion assertions reverted to the detail-keeps-main-0 contract; new tests cover limit-code detail rows and explicit-main-wins-over-later-details; apiserver convert_test.go gains the detail-only name-promotion cases Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
lei.chen
commit
0c8607345e
663 changed files with 139848 additions and 13614 deletions
|
|
@ -5504,9 +5504,15 @@ static void test_bug1475_catchup_survives_midstream_checkpoint(void)
|
|||
* synchronous flush right after so persist_durable_floor() (which the
|
||||
* ha.c bug-1475 fix reads) reflects it immediately rather than waiting on
|
||||
* the background fsync_worker's own timer (cfg.wal_sync_ms=5000). */
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_checkpoint(&leader->persist)); /* lsn=2 */
|
||||
/* bug-1870: checkpoints no longer write a marker; this test now covers a
|
||||
* WAL written by an older binary, whose checkpoint marker took lsn=2. */
|
||||
TEST_ASSERT_EQUAL_INT(0,
|
||||
persist_test_write_legacy_checkpoint_marker(&leader->persist)); /* lsn=2 */
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_checkpoint(&leader->persist));
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_flush(&leader->persist));
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_checkpoint(&recv->persist)); /* lsn=2 */
|
||||
TEST_ASSERT_EQUAL_INT(0,
|
||||
persist_test_write_legacy_checkpoint_marker(&recv->persist)); /* lsn=2 */
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_checkpoint(&recv->persist));
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_flush(&recv->persist));
|
||||
|
||||
/* bug-1466 (unfixed, deliberately NOT touched by this fix): the checkpoint
|
||||
|
|
@ -5565,6 +5571,79 @@ static void test_bug1475_catchup_survives_midstream_checkpoint(void)
|
|||
printf("test_bug1475_catchup_survives_midstream_checkpoint: PASS\n");
|
||||
}
|
||||
|
||||
/* bug-1870: a checkpoint taken by ONE member only -- the primary's periodic
|
||||
* max-wal-records checkpoint, or the one before a snapshot send -- must not
|
||||
* leave a hole in the replicated log. persist_write_checkpoint_marker used to
|
||||
* consume an LSN for its local-only WAL_CHECKPOINT marker, so the leader's
|
||||
* next real entry carried prev_log_index = the marker's LSN, which no standby
|
||||
* holds: logOk failed forever and writes stalled (--force-snapshot soak). */
|
||||
static void test_bug1870_leader_only_checkpoint_leaves_no_log_hole(void)
|
||||
{
|
||||
char ltmp[256], rtmp[256];
|
||||
strcpy(ltmp, "/tmp/xce_cr_b1870_leader_XXXXXX");
|
||||
strcpy(rtmp, "/tmp/xce_cr_b1870_recv_XXXXXX");
|
||||
TEST_ASSERT_NOT_NULL(mkdtemp(ltmp));
|
||||
TEST_ASSERT_NOT_NULL(mkdtemp(rtmp));
|
||||
|
||||
cluster_t *leader = t42_make_cluster(ltmp);
|
||||
cluster_t *recv = t42_make_cluster(rtmp);
|
||||
TEST_ASSERT_NOT_NULL(leader);
|
||||
TEST_ASSERT_NOT_NULL(recv);
|
||||
|
||||
enum { TERM = 3 };
|
||||
t42_make_leader(leader, TERM, /*n_peers=*/3);
|
||||
t_bug1475_make_follower(recv, TERM, /*n_peers=*/3);
|
||||
|
||||
/* identical prefix through lsn=1 on both */
|
||||
persist_set_term(&leader->persist, TERM);
|
||||
TEST_ASSERT_EQUAL_INT(0,
|
||||
persist_wal_async(&leader->persist, WAL_JOB_UPDATE, NULL, 0)); /* lsn=1 */
|
||||
leader->ha.raft.log_len = 1;
|
||||
persist_set_term(&recv->persist, TERM);
|
||||
TEST_ASSERT_EQUAL_INT(0,
|
||||
persist_wal_async(&recv->persist, WAL_JOB_UPDATE, NULL, 0)); /* lsn=1 */
|
||||
recv->ha.raft.log_len = 1;
|
||||
atomic_store_explicit(&recv->ha.applied_lsn, 1, memory_order_release);
|
||||
|
||||
/* ONLY the leader checkpoints; it must not take a log slot */
|
||||
uint64_t before = (uint64_t)leader->persist.current_lsn;
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_checkpoint(&leader->persist));
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_flush(&leader->persist));
|
||||
TEST_ASSERT_EQUAL_UINT64_MESSAGE(before,
|
||||
(uint64_t)leader->persist.current_lsn,
|
||||
"bug-1870: a checkpoint must not consume a replicated-log LSN");
|
||||
|
||||
/* leader appends two real entries after its checkpoint */
|
||||
persist_set_term(&leader->persist, TERM);
|
||||
TEST_ASSERT_EQUAL_INT(0,
|
||||
persist_wal_async(&leader->persist, WAL_JOB_UPDATE, NULL, 0));
|
||||
TEST_ASSERT_EQUAL_INT(0,
|
||||
persist_wal_async(&leader->persist, WAL_JOB_UPDATE, NULL, 0));
|
||||
uint64_t tip = (uint64_t)leader->persist.current_lsn - 1;
|
||||
atomic_store_explicit(&leader->persist.committed_lsn, tip,
|
||||
memory_order_release);
|
||||
|
||||
/* real catch-up streamer -> real receive path */
|
||||
g_bug1475_recv = recv;
|
||||
xce_wal_stream_request_hook = bug1475_replay_hook;
|
||||
int rc = ha_stream_wal_to_standby(/*ep=*/NULL, leader->persist.wal_path,
|
||||
&leader->persist, /*leader_term=*/TERM,
|
||||
/*standby_lsn=*/1, /*committed_lsn=*/tip);
|
||||
xce_wal_stream_request_hook = NULL;
|
||||
g_bug1475_recv = NULL;
|
||||
TEST_ASSERT_EQUAL_INT(0, rc);
|
||||
|
||||
TEST_ASSERT_EQUAL_UINT64_MESSAGE(tip,
|
||||
atomic_load_explicit(&recv->ha.applied_lsn, memory_order_acquire),
|
||||
"bug-1870: a standby that never checkpointed must still catch up "
|
||||
"past the leader's checkpoint");
|
||||
TEST_ASSERT_EQUAL_UINT64(tip, recv->ha.raft.log_len);
|
||||
TEST_ASSERT_EQUAL_UINT64(TERM, persist_entry_term(&recv->persist, tip));
|
||||
|
||||
t42_teardown(leader);
|
||||
t42_teardown(recv);
|
||||
}
|
||||
|
||||
/*
|
||||
* bug-1028 — the catch-up streamer must source each streamed entry's term and
|
||||
* the AppendEntries prev_log_term from the AUTHORITATIVE per-LSN term index
|
||||
|
|
@ -5816,7 +5895,10 @@ static void test_bug1475_finding2_snapshot_install_checkpoint_term_absent(void)
|
|||
TEST_ASSERT_EQUAL_UINT64(2, (uint64_t)leader->persist.current_lsn);
|
||||
atomic_store_explicit(&leader->persist.last_applied_lsn, 2,
|
||||
memory_order_release);
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_checkpoint(&leader->persist)); /* lsn=2 */
|
||||
/* bug-1870: legacy (pre-fix) marker at lsn=2, as an older binary wrote. */
|
||||
TEST_ASSERT_EQUAL_INT(0,
|
||||
persist_test_write_legacy_checkpoint_marker(&leader->persist)); /* lsn=2 */
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_checkpoint(&leader->persist));
|
||||
TEST_ASSERT_EQUAL_INT(0, persist_flush(&leader->persist));
|
||||
TEST_ASSERT_EQUAL_UINT64_MESSAGE(2, leader->persist.checkpoint_lsn,
|
||||
"precondition: checkpoint_lsn must coincide with the checkpoint "
|
||||
|
|
@ -6996,8 +7078,11 @@ static int cc_poll_flag(_Atomic int *flag, int timeout_ms)
|
|||
return atomic_load(flag);
|
||||
}
|
||||
|
||||
static void cc_run_offload_probe(int offload_on, const char *label)
|
||||
static void cc_run_offload_probe_ex(int offload_on, int start_lane,
|
||||
const char *label)
|
||||
{
|
||||
int saved_offload = ha_catchup_offload_enabled();
|
||||
|
||||
char tmpdir[256];
|
||||
snprintf(tmpdir, sizeof(tmpdir), "/tmp/xce_cc_%s_XXXXXX",
|
||||
offload_on ? "on" : "off");
|
||||
|
|
@ -7028,7 +7113,8 @@ static void cc_run_offload_probe(int offload_on, const char *label)
|
|||
.max_promotion_retries = 5,
|
||||
};
|
||||
TEST_ASSERT_EQUAL_INT(0, ha_init(&c.ha, &hcfg, 0));
|
||||
TEST_ASSERT_EQUAL_INT(0, slowlane_init(&c.slowlane, 2, 4));
|
||||
if (start_lane)
|
||||
TEST_ASSERT_EQUAL_INT(0, slowlane_init(&c.slowlane, 2, 4));
|
||||
TEST_ASSERT_EQUAL_INT(0, delaymgr_init(&c.delaymgr, 5, cc_run_delay_task));
|
||||
TEST_ASSERT_EQUAL_INT(0, delaymgr_start(&c.delaymgr));
|
||||
|
||||
|
|
@ -7051,7 +7137,7 @@ static void cc_run_offload_probe(int offload_on, const char *label)
|
|||
TEST_ASSERT_EQUAL_INT(0,
|
||||
delay_task_ms(&c.delaymgr, 1, 1, cc_probe_fn, &probe));
|
||||
|
||||
if (offload_on) {
|
||||
if (offload_on && start_lane) {
|
||||
TEST_ASSERT_TRUE_MESSAGE(cc_poll_flag(&probe.fired, 1000),
|
||||
"GREEN: with catch-up offloaded to the slow lane, the delaymgr "
|
||||
"worker thread must be free to run the NEXT queued task "
|
||||
|
|
@ -7079,11 +7165,12 @@ static void cc_run_offload_probe(int offload_on, const char *label)
|
|||
"the probe task must eventually fire once the gate is released");
|
||||
|
||||
ha_repl_stream_wal_hook = NULL;
|
||||
ha_set_catchup_offload_for_test(0); /* restore production default */
|
||||
ha_set_catchup_offload_for_test(saved_offload);
|
||||
cc_gate_destroy(&g_cc_gate);
|
||||
delaymgr_stop(&c.delaymgr);
|
||||
delaymgr_fini(&c.delaymgr);
|
||||
slowlane_fini(&c.slowlane);
|
||||
if (start_lane)
|
||||
slowlane_fini(&c.slowlane);
|
||||
ha_fini(&c.ha);
|
||||
persist_stop(&c.persist);
|
||||
persist_fini(&c.persist);
|
||||
|
|
@ -7091,6 +7178,21 @@ static void cc_run_offload_probe(int offload_on, const char *label)
|
|||
printf("%s: PASS\n", label);
|
||||
}
|
||||
|
||||
static void cc_run_offload_probe(int offload_on, const char *label)
|
||||
{
|
||||
cc_run_offload_probe_ex(offload_on, /*start_lane=*/1, label);
|
||||
}
|
||||
|
||||
/* bug-1451: with the offload ON (now the production default) and a slow
|
||||
* lane that was never started -- the lightweight fixtures that made the
|
||||
* offload hang when it was first flipped ON -- the catch-up body must still
|
||||
* run (inline), never sit forever in a lane with no workers. */
|
||||
static void test_catchup_offload_without_started_lane_runs_inline(void)
|
||||
{
|
||||
cc_run_offload_probe_ex(1, /*start_lane=*/0,
|
||||
"test_catchup_offload_without_started_lane_runs_inline");
|
||||
}
|
||||
|
||||
static void test_catchup_offload_probe_runs_promptly(void)
|
||||
{
|
||||
cc_run_offload_probe(1, "test_catchup_offload_probe_runs_promptly");
|
||||
|
|
@ -7454,6 +7556,7 @@ void test_config_repl(void)
|
|||
RUN_TEST(test_wal_stream_empty_range_no_batches);
|
||||
RUN_TEST(test_wal_stream_skips_checkpoint_marker);
|
||||
RUN_TEST(test_bug1475_catchup_survives_midstream_checkpoint); /* bug-1475 */
|
||||
RUN_TEST(test_bug1870_leader_only_checkpoint_leaves_no_log_hole); /* bug-1870 */
|
||||
RUN_TEST(test_wal_stream_term_from_index_not_raw_hdr); /* bug-1028 */
|
||||
/* Task 3.5: snapshot chunk transfer (sender side) */
|
||||
RUN_TEST(test_snapshot_transfer_two_chunks);
|
||||
|
|
@ -7480,6 +7583,7 @@ void test_config_repl(void)
|
|||
/* xce-audit-invariants controller pass, thread-role violation 2 fix
|
||||
* (2026-07-18): catch-up continuation slow-lane offload */
|
||||
RUN_TEST(test_catchup_offload_probe_runs_promptly);
|
||||
RUN_TEST(test_catchup_offload_without_started_lane_runs_inline);
|
||||
RUN_TEST(test_catchup_inline_probe_blocks_RED);
|
||||
/* bug-1476: concurrent load_primary_config() UAF/double-free race */
|
||||
RUN_TEST(test_bug1476_concurrent_load_primary_config_no_uaf_or_double_free);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue