feat(ha-convergence-recovery): commit-gated scheduler egress — submit_lsn tagging + watermark gate (task 1.2)

This commit is contained in:
Rongsong Shen 2026-06-14 02:33:36 +00:00
parent 7ff81156c2
commit b9ddfa60cb
9 changed files with 241 additions and 67 deletions

View file

@ -4,14 +4,14 @@
> Verify exact line numbers with grep before editing (this spec's research.md §1 has the 2026-06-11 verified anchors; files drift).
> Task 8 (E2E) additionally requires **ha-single-leader-role** to have landed and a live `/xce-deploy-ha` cluster.
- [ ] 1. Egress filter: submit_lsn tagging and commit-watermark gate on scheduler exports
- [x] 1. Egress filter: submit_lsn tagging and commit-watermark gate on scheduler exports
- [x] 1.1 Write RED tests for tagging and the scheduler-export watermark
- Create `testsuite/test_egress_filter.c` (XCE_TESTING, AAA structure) and register it in `controller_unit_test_src` (meson.build); add `uint64_t submit_lsn` to the `job_t` wrapper (types.h:483-485 anchor) and declare `job_submit_committed` / `collect_jobs_committed` in the dataex header so tests compile
- Tests: a live-submitted job is tagged with its WAL entry LSN; a job restored via `apply_wal_entry`/`restore_job` replay is tagged with the replayed entry's LSN; a job with `submit_lsn > committed_lsn` is absent from `collect_jobs_committed` results and from the JOB_NEW incremental push list; the same job appears once `committed_lsn` advances to cover it; `submit_lsn == 0` is always exportable; plain `collect_jobs` (status/API path) still returns the uncommitted job
- Observable completion: build succeeds with the new field/header declarations; all new test cases FAIL (red) against unimplemented predicate/tagging
- _Requirements: 3.1, 3.2, 3.3, 3.4_
- [ ] 1.2 Implement tagging and the watermark gate to green
- [x] 1.2 Implement tagging and the watermark gate to green
- Tagging: `persist_job_hook_fun` (persist.c:1107-1125 anchor) sets `submit_lsn = written_lsn` on each submitted job after a successful append; `apply_wal_entry` (persist.c:729-731 anchor) passes `hdr->lsn` through to `restore_job` (job.c:71), which tags the restored job
- Predicate + wrapper in dataex.c: `job_submit_committed` = `submit_lsn == 0 || submit_lsn <= committed_lsn` (one atomic load, lock-free); `collect_jobs_committed` wraps `collect_jobs` with the predicate
- Scheduler-bound consumers only: `xcesched_push_jobs` (sched.c:183 anchor) collects via the committed-only wrapper; `xcesched_push_joblist` (JOB_NEW push fed by `sched_job_new_fun`, sched.c:236-248 anchor) filters its list with the predicate; apiserver.c:83 and opget.c:173 remain on unfiltered `collect_jobs`
@ -91,3 +91,10 @@
- Observable completion: all E2E assertions hold on a live `/xce-deploy-ha` cluster; results recorded in completion notes
- _Depends: 7_
- _Requirements: 2.4, 9.1, 9.2, 8.1, 8.2, 3.1_
## Implementation Notes
- **Task 1.2 — predicate placed in new `egress.c`, not `dataex.c`** (design D3 "refinement, not contradiction"): `dataex.c` defines `notify()` (dataex.c:384) and `collect_confvars()`, both already stubbed in the unit-test binary (test_commit_barrier_reply.c, test_worker_join.c). Linking `dataex.c` into `controller_unit_test_src` is a duplicate-symbol error, so `job_submit_committed`/`collect_jobs_committed` live in a dedicated TU `src/controller/egress.c`, registered in BOTH `controller_src` and `controller_unit_test_src`. Header decls remain in `cluster.h` (the controller's "dataex header" — there is no `dataex.h`). Any future scheduler-bound export path adopts the predicate from there.
- **Test-link rule**: a production symbol that the unit-test suite already stubs cannot have its owning `.c` added to `controller_unit_test_src`. Factor newly-testable logic into its own TU instead.
- **`restore_job` gained an `entry_lsn` parameter** (replay tagging). Callers updated: cluster.h decl, job.c def, persist.c extern + apply_wal_entry caller, test_repl_apply.c stub. Any new caller must pass the WAL entry header LSN (`hdr->lsn`), or `0` if unknown (0 = exportable).
- **Build note**: the full meson controller suite SIGTERMs at the 300s wall-clock cap on the slow commit-barrier timeout suites (pre-existing). For per-task GREEN confirmation, run `controller_unit_test -t <suite>` directly with `ASAN_OPTIONS=detect_leaks=0` (harness teardown leaks are a pre-existing accepted pattern; the meson gate sets the same option).