feat(raft-apply-decoupling): split fact receipt to durable-append-only (task 4.1)
Convert on_job_start/on_job_finish/on_job_fail/on_job_postdone to validate and durably append only -- BEFORE_EVENT validation hook unchanged, AFTER_EVENT (post-mutation) hook no longer fires at receipt, live state mutation removed. Adds persist_submit_job_fact() (persist.c) since the durable append previously lived inside the very AFTER_EVENT hook this task silences; wire format is unchanged and identical to what apply_committed_entry() already routes. mark_job_start/mark_job_finish (single caller each) renamed in place to record_job_start_fact/record_job_finish_fact. mark_job_fail/ mark_job_update (shared with job_ctrl's client-command path, out of boundary) left untouched; new durable-append-only siblings record_job_fail_fact/record_job_update_fact added instead. Accepts a narrow, permanent regression: on_job_postdone's finish_time anomaly backstop has no wire-format equivalent and is dropped (affects only a rare no-prior-exit-status path). Two critical hazards discovered by review, documented in tasks.md, MUST be resolved before task 6 wires apply_committed_entry() into a live path: (1) persist_job_hook_fun still fires on HOOK_DETAIL_JOB_UPDATE and will double-append every live-applied fact; (2) record_job_fail_fact/ record_job_update_fact precompute a final state that can be silently clobbered by a concurrent intervening fact once apply is deferred by the commit round-trip.
This commit is contained in:
parent
f357215a6d
commit
23cbfb48e1
5 changed files with 492 additions and 116 deletions
|
|
@ -2078,61 +2078,61 @@ static void test_fact_retention_across_quorum_loss(void)
|
|||
}
|
||||
|
||||
/* ════════════════════════════════════════════════════════════════════════
|
||||
* Idempotency / ordering guards for mark_job_* (task 6.1, Req 5.1/5.2/5.3)
|
||||
* Idempotency / ordering guards for the fact-transition receipt handlers
|
||||
* (guard extraction: task 2.1, Req 5.1/5.2/5.3 — landed and green)
|
||||
* ════════════════════════════════════════════════════════════════════════
|
||||
*
|
||||
* RED-phase tests for the transition guards that task 6.2 adds at the top
|
||||
* of mark_job_start / mark_job_finish / mark_job_fail / mark_job_update
|
||||
* (job.c:583/638/696/726 pre-task anchors). Design §Idempotency Guards:
|
||||
* - duplicate (already-reflected transition) → INFO log, NO-OP: state
|
||||
* unchanged, BEFORE/AFTER_EVENT hooks NOT fired → NO WAL entry appended
|
||||
* (current_lsn unmoved), caller-visible rc treated as success-no-op (5.1)
|
||||
* Pins the transition guards task 2.1 extracted into apply_job_start_
|
||||
* transition / apply_job_finish_transition / apply_job_fail_transition /
|
||||
* apply_job_update_transition, and peeked at the top of the receipt-side
|
||||
* wrappers (record_job_start_fact / record_job_finish_fact — renamed from
|
||||
* mark_job_start / mark_job_finish by task 4.1 — and mark_job_fail /
|
||||
* mark_job_update, still used by job_ctrl's separate command path). Design
|
||||
* §Idempotency Guards:
|
||||
* - duplicate (already-reflected transition) → INFO log, NO-OP: no WAL
|
||||
* entry appended (current_lsn unmoved), caller-visible rc treated as
|
||||
* success-no-op (5.1)
|
||||
* - backward (terminal current state, non-terminal requested: START on a
|
||||
* finished job, UPDATE clearing terminal bits) → WARN log, REJECT,
|
||||
* terminal state preserved (5.2)
|
||||
* - replay: re-applying the same transition converges to single-
|
||||
* application state — no second WAL entry, identical final state (5.3)
|
||||
* - mark_job_fail's existing fail-after-finish PERR-merge (job.c:702-716)
|
||||
* is forward-compatible and must be PRESERVED by the guard (pin)
|
||||
* finished job, UPDATE clearing terminal bits) → WARN log, REJECT, no
|
||||
* WAL entry appended (5.2)
|
||||
* - replay: re-applying the same transition appends no second WAL entry
|
||||
* (5.3)
|
||||
* - apply_job_fail_transition's existing fail-after-finish PERR-merge is
|
||||
* forward-compatible and must be PRESERVED by the guard (pin)
|
||||
*
|
||||
* Drivers: the static mark_job_* are reached through their non-static
|
||||
* callers on_job_start / on_job_finish / on_job_fail (job.c) — exactly the
|
||||
* Drivers: on_job_start / on_job_finish / on_job_fail (job.c) — exactly the
|
||||
* fact path used by test_fact_retention_across_quorum_loss, minus the
|
||||
* opcode decode. These wrappers each return 0 unconditionally today, so the
|
||||
* opcode decode. These wrappers each return 0 unconditionally, so the
|
||||
* "success-no-op rc" observable (every caller treats a guarded-out
|
||||
* transition as success) is asserted directly on the wrapper rc.
|
||||
*
|
||||
* Observability of state: the per-job WAL append count and the LAST
|
||||
* recorded post-transition state come from cbrp_count_jobstate_entries
|
||||
* (direct on-disk scan, defined above); the job's LIVE in-memory state is
|
||||
* read via cbrp_job_live_state (key_find on g_hc.jobmgr). current_lsn is
|
||||
* the "did a hook fire / WAL entry append" oracle (a guarded-out no-op runs
|
||||
* no hooks → no persist_wal_async → current_lsn frozen).
|
||||
* Observability: the per-job WAL append count and the LAST recorded
|
||||
* post-transition state come from cbrp_count_jobstate_entries (direct
|
||||
* on-disk scan, defined above) — current_lsn is the "did a WAL entry
|
||||
* append" oracle (a guarded-out no-op never calls persist_wal_async →
|
||||
* current_lsn frozen). cbrp_job_live_state remains meaningful in these
|
||||
* tests only because cbrp_drive_to_done (task 4.1) now reaches its
|
||||
* terminal precondition by applying the guard cores directly, as a
|
||||
* test-only stand-in for the not-yet-built live apply pump — since task 4.1,
|
||||
* the receipt handlers themselves never mutate live state on ANY path
|
||||
* (guarded-out or accepted), so a passing guard test no longer implies
|
||||
* "the guard blocked a live mutation," only "the guard blocked a WAL
|
||||
* append." test_guard_forward_update_on_terminal_preserved and
|
||||
* test_guard_fail_after_finish_preserves_perr_merge were updated by task 4.1
|
||||
* to assert the forward-merge/PERR-merge outcome against the WAL-recorded
|
||||
* fact instead of live state, plus an explicit live-state-unchanged pin.
|
||||
*
|
||||
* RED expectation against the UNGUARDED mark_job_* (no guard yet):
|
||||
* - duplicate FINISH re-fires the AFTER_EVENT hook → a 2nd WAL_JOB_FINISH
|
||||
* append, current_lsn advances → the "current_lsn unmoved / count==1"
|
||||
* assertions FAIL.
|
||||
* - START on a DONE job unconditionally sets state = JOB_STAT_RUN and
|
||||
* appends WAL_JOB_START → the "state preserved DONE / no new append"
|
||||
* assertions FAIL.
|
||||
* - replay FINISH appends a duplicate entry → the "single append"
|
||||
* assertion FAILS.
|
||||
* The fail-after-finish PERR-merge pin and the success-no-op rc pins lock
|
||||
* behavior that must hold both before and after the guard (preservation
|
||||
* pins, like the task-3.1 validation pins); they ride this group so 6.2
|
||||
* cannot silently regress the merge or the wrapper return contract.
|
||||
*
|
||||
* NOTE on the UPDATE-clears-terminal arm (5.2): mark_job_update is static
|
||||
* with no public caller that produces a terminal→non-terminal state
|
||||
* (on_job_postdone's update is forward — it OR's PDONE, preserving terminal;
|
||||
* job_ctrl SIGCONT only clears suspend bits). mark_job_start IS the
|
||||
* canonical "set state to the non-terminal JOB_STAT_RUN" overwrite, so the
|
||||
* START-on-terminal case below is the faithful, publicly-reachable witness
|
||||
* of the shared backward rule the design lists for START and UPDATE
|
||||
* together. test_guard_forward_update_on_terminal_preserved additionally
|
||||
* pins that a FORWARD update on a terminal job (on_job_postdone) is NOT
|
||||
* over-rejected — the boundary the guard must respect.
|
||||
* NOTE on the UPDATE-clears-terminal arm (5.2): mark_job_update/
|
||||
* record_job_update_fact have no public caller that produces a
|
||||
* terminal→non-terminal state (on_job_postdone's update is forward — it
|
||||
* OR's PDONE, preserving terminal; job_ctrl SIGCONT only clears suspend
|
||||
* bits). record_job_start_fact IS the canonical "set state to the
|
||||
* non-terminal JOB_STAT_RUN" overwrite, so the START-on-terminal case below
|
||||
* is the faithful, publicly-reachable witness of the shared backward rule
|
||||
* the design lists for START and UPDATE together.
|
||||
* test_guard_forward_update_on_terminal_preserved additionally pins that a
|
||||
* FORWARD update on a terminal job (on_job_postdone) is NOT over-rejected —
|
||||
* the boundary the guard must respect.
|
||||
*/
|
||||
|
||||
/* Read a job's live in-memory state from g_hc.jobmgr (post-transition).
|
||||
|
|
@ -2160,13 +2160,27 @@ static int cbrp_job_live_state(uint64_t jobid)
|
|||
|
||||
/* Drive a job to terminal DONE via start(jobid) → finish(jobid, exit 0).
|
||||
* Returns current_lsn after the two forward facts (the watermark the
|
||||
* duplicate/backward cases must NOT advance). */
|
||||
* duplicate/backward cases must NOT advance).
|
||||
*
|
||||
* raft-apply-decoupling task 4.1: on_job_start/on_job_finish are now
|
||||
* durable-append-only — they no longer mutate the job's LIVE state (that
|
||||
* moves to apply_committed_entry(), driven by the not-yet-built live apply
|
||||
* pump, task 6). This helper's callers still need the job's LIVE state to
|
||||
* reach a genuine terminal DONE so they can exercise backward-reject /
|
||||
* duplicate-noop / forward-merge guard behavior against a real terminal
|
||||
* job, so — after driving the two facts through the real receipt path to
|
||||
* verify the durable append — this also applies the identical two
|
||||
* transitions directly through the task-2.1 guard cores
|
||||
* (apply_job_start_transition/apply_job_finish_transition), a test-only
|
||||
* stand-in for what the future apply pump will do live once an entry
|
||||
* commits. This is not a production code path. */
|
||||
static uint64_t cbrp_drive_to_done(uint64_t jobid)
|
||||
{
|
||||
static unsigned int gd_pids[1] = { 9999 };
|
||||
job_pidinfo_t start;
|
||||
xce_jobfinish_t fin;
|
||||
uint64_t lsn;
|
||||
job_t* job = NULL;
|
||||
|
||||
memset(&start, 0, sizeof(start));
|
||||
start.jobid = jobid;
|
||||
|
|
@ -2179,6 +2193,21 @@ static uint64_t cbrp_drive_to_done(uint64_t jobid)
|
|||
fin.exit_code = 0; /* → JOB_STAT_DONE */
|
||||
TEST_ASSERT_EQUAL_INT(0, on_job_finish(&g_hc, &fin));
|
||||
|
||||
lock(g_hc.jobmgr, RTTI_LOCK_READ);
|
||||
key_find(g_hc.jobmgr, (void *)jobid, (void **)&job);
|
||||
if (job != NULL) {
|
||||
reference(job);
|
||||
}
|
||||
unlock(g_hc.jobmgr);
|
||||
TEST_ASSERT_NOT_NULL_MESSAGE(job,
|
||||
"job must exist in jobmgr after cbrp_make_job + on_job_start");
|
||||
|
||||
lock(job, RTTI_LOCK_WRITE);
|
||||
apply_job_start_transition(job, &start);
|
||||
apply_job_finish_transition(job, &fin);
|
||||
unlock(job);
|
||||
dereference(job);
|
||||
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(JOB_STAT_DONE, cbrp_job_live_state(jobid),
|
||||
"precondition: job must be DONE after start→finish(0)");
|
||||
|
||||
|
|
@ -2238,9 +2267,10 @@ static void test_guard_duplicate_finish_on_done_is_noop(void)
|
|||
/* 5.2 — START on a finished (DONE) job is rejected: the terminal state is
|
||||
* preserved (never overwritten with JOB_STAT_RUN), NO WAL_JOB_START entry is
|
||||
* appended, and on_job_start returns a success-no-op code that the caller
|
||||
* (do_notify) treats as success. mark_job_start is the canonical
|
||||
* terminal→non-terminal overwrite, so this also witnesses the UPDATE-clears-
|
||||
* terminal arm of the shared backward rule (5.2). */
|
||||
* (do_notify) treats as success. record_job_start_fact (renamed from
|
||||
* mark_job_start by task 4.1) is the canonical terminal→non-terminal
|
||||
* overwrite, so this also witnesses the UPDATE-clears-terminal arm of the
|
||||
* shared backward rule (5.2). */
|
||||
static void test_guard_start_on_terminal_rejected_state_preserved(void)
|
||||
{
|
||||
static unsigned int rs_pids[1] = { 7001 };
|
||||
|
|
@ -2341,15 +2371,25 @@ static void test_guard_replay_finish_equals_single_application(void)
|
|||
}
|
||||
|
||||
/* 5.2 boundary pin — a FORWARD update on a terminal job must NOT be
|
||||
* over-rejected by the guard. on_job_postdone is the reachable mark_job_update
|
||||
* caller: on a DONE job it OR's JOB_STAT_PDONE (terminal preserved, strictly
|
||||
* forward). This must still apply (state gains PDONE, an entry appends) after
|
||||
* the guard lands — the guard rejects backward, never a forward merge.
|
||||
* Preservation pin: green before and after task 6.2. */
|
||||
* over-rejected by the guard. on_job_postdone is the reachable
|
||||
* record_job_update_fact caller: on a DONE job it OR's JOB_STAT_PDONE
|
||||
* (terminal preserved, strictly forward). This must still durably append
|
||||
* (an entry appends, carrying the merged state) — the guard rejects
|
||||
* backward, never a forward merge. Preservation pin: green before and after
|
||||
* task 6.2.
|
||||
*
|
||||
* raft-apply-decoupling task 4.1: on_job_postdone is now durable-append-only
|
||||
* — the oracle for "did the forward merge compute correctly" moves from live
|
||||
* state to the WAL-recorded fact (cbrp_count_jobstate_entries), mirroring
|
||||
* test_fact_retention_across_quorum_loss. The live-state-unchanged
|
||||
* assertion is this task's own new pin (Req 5.1): live state must still show
|
||||
* only DONE (from cbrp_drive_to_done's direct apply), never PDONE, since
|
||||
* nothing applies this fact live yet. */
|
||||
static void test_guard_forward_update_on_terminal_preserved(void)
|
||||
{
|
||||
uint64_t lsn_after_done, lsn_after_pdone;
|
||||
int st;
|
||||
uint32_t st_disk = 0;
|
||||
int st_live;
|
||||
|
||||
cbrp_setup(3000);
|
||||
cbrp_make_job(73, &g_job_submit);
|
||||
|
|
@ -2363,31 +2403,54 @@ static void test_guard_forward_update_on_terminal_preserved(void)
|
|||
pthread_mutex_lock(&g_hc.persist.lock);
|
||||
lsn_after_pdone = g_hc.persist.current_lsn;
|
||||
pthread_mutex_unlock(&g_hc.persist.lock);
|
||||
st = cbrp_job_live_state(73);
|
||||
|
||||
/* Assert: terminal preserved, PDONE merged forward, the update appended */
|
||||
TEST_ASSERT_TRUE_MESSAGE((st & JOB_STAT_DONE) != 0,
|
||||
"a forward update on a terminal job must preserve the terminal bit");
|
||||
TEST_ASSERT_TRUE_MESSAGE((st & JOB_STAT_PDONE) != 0,
|
||||
"the forward update must merge JOB_STAT_PDONE (not be rejected)");
|
||||
/* Assert: the update appended, carrying the correctly forward-merged
|
||||
* state (terminal DONE preserved, PDONE merged in — not rejected). */
|
||||
TEST_ASSERT_TRUE_MESSAGE(lsn_after_pdone > lsn_after_done,
|
||||
"a forward update on a terminal job must still append its WAL entry "
|
||||
"— the guard rejects backward transitions, never forward merges");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(1,
|
||||
cbrp_count_jobstate_entries(&g_hc.persist, WAL_JOB_UPDATE, 73,
|
||||
&st_disk),
|
||||
"exactly one UPDATE fact must be durably recorded (Req 5.1)");
|
||||
TEST_ASSERT_TRUE_MESSAGE((st_disk & JOB_STAT_DONE) != 0,
|
||||
"the recorded UPDATE fact must preserve the terminal bit");
|
||||
TEST_ASSERT_TRUE_MESSAGE((st_disk & JOB_STAT_PDONE) != 0,
|
||||
"the recorded UPDATE fact must merge JOB_STAT_PDONE (not be "
|
||||
"rejected)");
|
||||
|
||||
/* Assert (Req 5.1, task 4.1's own observable): live state is UNCHANGED
|
||||
* by the receipt handler — still just DONE, no PDONE — since nothing
|
||||
* applies this fact live until the apply pump (task 6) exists. */
|
||||
st_live = cbrp_job_live_state(73);
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(JOB_STAT_DONE, st_live,
|
||||
"on_job_postdone must leave live job state unchanged (Req 5.1) — "
|
||||
"the PDONE merge above is recorded in the WAL only, not applied "
|
||||
"live, until the apply pump exists");
|
||||
|
||||
cbrp_teardown();
|
||||
printf("test_guard_forward_update_on_terminal_preserved: PASS\n");
|
||||
}
|
||||
|
||||
/* 5.2 pin — mark_job_fail's existing fail-after-finish PERR-merge
|
||||
* (job.c:702-716) must be PRESERVED by the guard: a POST_EXEC-stage fail on a
|
||||
* DONE job keeps DONE and OR's JOB_STAT_PERR (it does NOT reset the job to a
|
||||
* bare JOB_STAT_EXIT). Forward-compatible terminal→terminal merge — the guard
|
||||
* must not reject it. Preservation pin: green before and after task 6.2. */
|
||||
/* 5.2 pin — apply_job_fail_transition's existing fail-after-finish PERR-merge
|
||||
* must be PRESERVED: a POST_EXEC-stage fail on a DONE job keeps DONE and
|
||||
* OR's JOB_STAT_PERR (it does NOT reset the job to a bare JOB_STAT_EXIT).
|
||||
* Forward-compatible terminal→terminal merge — the guard must not reject it.
|
||||
* Preservation pin: green before and after task 6.2.
|
||||
*
|
||||
* raft-apply-decoupling task 4.1: on_job_fail (via the new
|
||||
* record_job_fail_fact) is now durable-append-only — it computes the
|
||||
* PERR-merged target state on a throwaway copy of the job and durably
|
||||
* appends it, without touching the real job's live state. The oracle for
|
||||
* "did the merge compute correctly" moves from live state to the
|
||||
* WAL-recorded fact (cbrp_count_jobstate_entries); the live-state-unchanged
|
||||
* assertion is this task's own new pin (Req 5.1). */
|
||||
static void test_guard_fail_after_finish_preserves_perr_merge(void)
|
||||
{
|
||||
job_failinfo_t failinfo;
|
||||
uint64_t lsn_after_done, lsn_after_fail;
|
||||
int st;
|
||||
uint32_t st_disk = 0;
|
||||
int st_live;
|
||||
|
||||
cbrp_setup(3000);
|
||||
cbrp_make_job(74, &g_job_submit);
|
||||
|
|
@ -2406,23 +2469,175 @@ static void test_guard_fail_after_finish_preserves_perr_merge(void)
|
|||
pthread_mutex_lock(&g_hc.persist.lock);
|
||||
lsn_after_fail = g_hc.persist.current_lsn;
|
||||
pthread_mutex_unlock(&g_hc.persist.lock);
|
||||
st = cbrp_job_live_state(74);
|
||||
|
||||
/* Assert: DONE preserved, PERR merged in — not reset to bare EXIT */
|
||||
TEST_ASSERT_TRUE_MESSAGE((st & JOB_STAT_DONE) != 0,
|
||||
"fail-after-finish must keep the DONE terminal bit (PERR-merge "
|
||||
"behavior, job.c:702-716)");
|
||||
TEST_ASSERT_TRUE_MESSAGE((st & JOB_STAT_PERR) != 0,
|
||||
"fail-after-finish must OR JOB_STAT_PERR onto the existing terminal "
|
||||
"state (PERR-merge behavior, job.c:702-716)");
|
||||
/* Assert: the FAIL fact appended, carrying the correctly PERR-merged
|
||||
* state — DONE preserved, PERR merged in, not reset to bare EXIT. */
|
||||
TEST_ASSERT_TRUE_MESSAGE(lsn_after_fail > lsn_after_done,
|
||||
"the PERR-merge fail must still append its WAL entry (forward "
|
||||
"terminal→terminal merge — not a guarded-out no-op)");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(1,
|
||||
cbrp_count_jobstate_entries(&g_hc.persist, WAL_JOB_FAIL, 74,
|
||||
&st_disk),
|
||||
"exactly one FAIL fact must be durably recorded (Req 5.1)");
|
||||
TEST_ASSERT_TRUE_MESSAGE((st_disk & JOB_STAT_DONE) != 0,
|
||||
"the recorded FAIL fact must keep the DONE terminal bit "
|
||||
"(PERR-merge behavior)");
|
||||
TEST_ASSERT_TRUE_MESSAGE((st_disk & JOB_STAT_PERR) != 0,
|
||||
"the recorded FAIL fact must OR JOB_STAT_PERR onto the existing "
|
||||
"terminal state (PERR-merge behavior)");
|
||||
|
||||
/* Assert (Req 5.1, task 4.1's own observable): live state is UNCHANGED
|
||||
* by the receipt handler — still just DONE, no PERR — since nothing
|
||||
* applies this fact live until the apply pump (task 6) exists. */
|
||||
st_live = cbrp_job_live_state(74);
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(JOB_STAT_DONE, st_live,
|
||||
"on_job_fail must leave live job state unchanged (Req 5.1) — the "
|
||||
"PERR merge above is recorded in the WAL only, not applied live, "
|
||||
"until the apply pump exists");
|
||||
|
||||
cbrp_teardown();
|
||||
printf("test_guard_fail_after_finish_preserves_perr_merge: PASS\n");
|
||||
}
|
||||
|
||||
/* ════════════════════════════════════════════════════════════════════════
|
||||
* raft-apply-decoupling task 4.1 (Req 5.1) — receipt handlers durably
|
||||
* append only, no live mutation, no AFTER_EVENT firing
|
||||
* ════════════════════════════════════════════════════════════════════════
|
||||
*
|
||||
* Spy hook counting BEFORE_EVENT vs AFTER_EVENT firings for
|
||||
* XCE_CONTROLLER_HOOK_JOB — registered alongside persist_start's production
|
||||
* hooks (persist_job_hook_fun etc., a fan-out, so both subscribers coexist).
|
||||
* Used to pin: the pre-mutation validation hook (BEFORE_EVENT) still fires
|
||||
* exactly once per accepted fact, unchanged; AFTER_EVENT never fires at
|
||||
* receipt time any more (it now only fires from apply_committed_entry(),
|
||||
* which nothing drives yet — task 6). */
|
||||
static int g_cbrp_before_fires;
|
||||
static int g_cbrp_after_fires;
|
||||
|
||||
static int cbrp_job_hook_spy(int type, int situation, int detail, void *data,
|
||||
va_list ap)
|
||||
{
|
||||
(void)type; (void)detail; (void)data; (void)ap;
|
||||
if (situation == HOOK_BEFORE_EVENT)
|
||||
g_cbrp_before_fires++;
|
||||
else if (situation == HOOK_AFTER_EVENT)
|
||||
g_cbrp_after_fires++;
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* The task's own RED/GREEN observable: after each of on_job_start/
|
||||
* on_job_finish/on_job_fail/on_job_postdone returns, the job's LIVE
|
||||
* in-memory state is unchanged — even though every one of the four facts
|
||||
* durably appends with the correct target state, and the pre-mutation
|
||||
* BEFORE_EVENT hook still fires exactly once per accepted fact. Against the
|
||||
* pre-task-4.1 code (mutate-then-notify at receipt) this test's live-state-
|
||||
* unchanged assertions fail on the very first fact (START moves live state
|
||||
* PEND -> RUN immediately) — confirmed by running this test against a
|
||||
* revert of job.c/persist.c/persist.h before applying this task's fix. */
|
||||
static void test_receipt_facts_durably_append_without_live_mutation(void)
|
||||
{
|
||||
static unsigned int rf_pids[1] = { 6001 };
|
||||
job_pidinfo_t start;
|
||||
xce_jobfinish_t fin;
|
||||
job_failinfo_t failinfo;
|
||||
uint32_t st_disk = 0;
|
||||
int live_before, live_after;
|
||||
|
||||
cbrp_setup(3000);
|
||||
cbrp_make_job(90, &g_job_submit);
|
||||
memset(&g_job_submit, 0, sizeof(g_job_submit));
|
||||
|
||||
g_cbrp_before_fires = 0;
|
||||
g_cbrp_after_fires = 0;
|
||||
TEST_ASSERT_EQUAL_INT(0,
|
||||
xce_add_hook(&g_hc.hooksys, XCE_CONTROLLER_HOOK_JOB, cbrp_job_hook_spy,
|
||||
NULL));
|
||||
|
||||
/* START */
|
||||
live_before = cbrp_job_live_state(90);
|
||||
memset(&start, 0, sizeof(start));
|
||||
start.jobid = 90;
|
||||
start.npids = 1;
|
||||
start.pids = &rf_pids[0];
|
||||
TEST_ASSERT_EQUAL_INT(0, on_job_start(&g_hc, &start));
|
||||
live_after = cbrp_job_live_state(90);
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(live_before, live_after,
|
||||
"on_job_start must leave live job state unchanged (Req 5.1)");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(1,
|
||||
cbrp_count_jobstate_entries(&g_hc.persist, WAL_JOB_START, 90,
|
||||
&st_disk),
|
||||
"on_job_start must durably append exactly one START fact (Req 5.1)");
|
||||
TEST_ASSERT_EQUAL_UINT32(JOB_STAT_RUN, st_disk);
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(1, g_cbrp_before_fires,
|
||||
"the pre-mutation validation hook (BEFORE_EVENT) must still fire "
|
||||
"unchanged at receipt time");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(0, g_cbrp_after_fires,
|
||||
"AFTER_EVENT must NOT fire at receipt time any more (Req 5.1, 5.3)");
|
||||
|
||||
/* FINISH */
|
||||
live_before = cbrp_job_live_state(90);
|
||||
memset(&fin, 0, sizeof(fin));
|
||||
fin.jobid = 90;
|
||||
fin.exit_code = 0;
|
||||
TEST_ASSERT_EQUAL_INT(0, on_job_finish(&g_hc, &fin));
|
||||
live_after = cbrp_job_live_state(90);
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(live_before, live_after,
|
||||
"on_job_finish must leave live job state unchanged (Req 5.1)");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(1,
|
||||
cbrp_count_jobstate_entries(&g_hc.persist, WAL_JOB_FINISH, 90,
|
||||
&st_disk),
|
||||
"on_job_finish must durably append exactly one FINISH fact (Req "
|
||||
"5.1)");
|
||||
TEST_ASSERT_EQUAL_UINT32(JOB_STAT_DONE, st_disk);
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(2, g_cbrp_before_fires,
|
||||
"BEFORE_EVENT must fire again for the FINISH fact");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(0, g_cbrp_after_fires,
|
||||
"AFTER_EVENT must still never fire at receipt time");
|
||||
|
||||
/* FAIL */
|
||||
live_before = cbrp_job_live_state(90);
|
||||
memset(&failinfo, 0, sizeof(failinfo));
|
||||
failinfo.jobid = 90;
|
||||
failinfo.stage = JOB_EXEC_STAGE_POST_EXEC;
|
||||
failinfo.code = 1;
|
||||
failinfo.detail = (char *)"synthetic fail for no-mutation pin";
|
||||
TEST_ASSERT_EQUAL_INT(0, on_job_fail(&g_hc, &failinfo));
|
||||
live_after = cbrp_job_live_state(90);
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(live_before, live_after,
|
||||
"on_job_fail must leave live job state unchanged (Req 5.1)");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(1,
|
||||
cbrp_count_jobstate_entries(&g_hc.persist, WAL_JOB_FAIL, 90,
|
||||
&st_disk),
|
||||
"on_job_fail must durably append exactly one FAIL fact (Req 5.1)");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(3, g_cbrp_before_fires,
|
||||
"BEFORE_EVENT must fire again for the FAIL fact — mark_job_fail has "
|
||||
"no guard, so it always fires");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(0, g_cbrp_after_fires,
|
||||
"AFTER_EVENT must still never fire at receipt time");
|
||||
|
||||
/* UPDATE (via on_job_postdone) */
|
||||
live_before = cbrp_job_live_state(90);
|
||||
TEST_ASSERT_EQUAL_INT(0, on_job_postdone(&g_hc, &g_req_ep, 90));
|
||||
live_after = cbrp_job_live_state(90);
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(live_before, live_after,
|
||||
"on_job_postdone must leave live job state unchanged (Req 5.1)");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(1,
|
||||
cbrp_count_jobstate_entries(&g_hc.persist, WAL_JOB_UPDATE, 90,
|
||||
&st_disk),
|
||||
"on_job_postdone must durably append exactly one UPDATE fact (Req "
|
||||
"5.1)");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(4, g_cbrp_before_fires,
|
||||
"BEFORE_EVENT must fire again for the UPDATE fact");
|
||||
TEST_ASSERT_EQUAL_INT_MESSAGE(0, g_cbrp_after_fires,
|
||||
"AFTER_EVENT must NEVER fire at receipt time for any of the four "
|
||||
"fact types (Req 5.1, 5.3) — it now only fires from "
|
||||
"apply_committed_entry(), which nothing drives live until task 6");
|
||||
|
||||
cbrp_teardown();
|
||||
printf("test_receipt_facts_durably_append_without_live_mutation: "
|
||||
"PASS\n");
|
||||
}
|
||||
|
||||
/* ── Test group entry point ──────────────────────────────────────────── */
|
||||
|
||||
void test_commit_barrier_reply(void)
|
||||
|
|
@ -2470,14 +2685,15 @@ void test_commit_barrier_reply(void)
|
|||
/* Fact retention across quorum loss (task 4.3, Req 3.1, 3.3, 3.4) —
|
||||
* green on landing against tasks 1.2 + 4.2 output */
|
||||
RUN_TEST(test_fact_retention_across_quorum_loss);
|
||||
/* mark_job_* idempotency / ordering guards (task 6.1, Req 5.1/5.2/5.3)
|
||||
* — the no-op / backward-reject / replay cases FAIL (red) against the
|
||||
* unguarded mark_job_*; the forward-merge and PERR-merge cases are
|
||||
* preservation pins (green before and after task 6.2) */
|
||||
/* Fact-transition idempotency / ordering guards (extracted by task 2.1,
|
||||
* already landed and green — the stale "RED-phase for task 6.2" framing
|
||||
* this comment block used to carry predates that landing) */
|
||||
RUN_TEST(test_guard_duplicate_finish_on_done_is_noop);
|
||||
RUN_TEST(test_guard_start_on_terminal_rejected_state_preserved);
|
||||
RUN_TEST(test_guard_replay_finish_equals_single_application);
|
||||
RUN_TEST(test_guard_forward_update_on_terminal_preserved);
|
||||
RUN_TEST(test_guard_fail_after_finish_preserves_perr_merge);
|
||||
/* Fact receipt handlers durably append only (task 4.1, Req 5.1) */
|
||||
RUN_TEST(test_receipt_facts_durably_append_without_live_mutation);
|
||||
UNITY_END();
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue