docs(buglog): bug-1921 fixed

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.wolf/bugs/bug-1921.md

@@ -1,8 +1,8 @@
 ---
 id: "bug-1921"
-status: "open"
-status_updated: "2026-10-05"
-status_note: "newly logged"
+status: "fixed"
+status_updated: "2026-10-06"
+status_note: ""
 component: "harness"
 severity: "wrong-result"
 timestamp: "2026-10-05T05:53:10Z"
@@ -11,6 +11,7 @@ occurrences: 1
 files: ["src/harness/test.c", "src/lib/xcectx.c"]
 tags: ["harness", "worker", "reconnect", "rusage"]
 related: ["bug-1917", "bug-1736"]
+fix_commits: ["70d56a5d"]
 gitea_issue: 345
 gitea_synced_status: "open"
 gitea_synced_hash: "3376844d91f7"
@@ -52,15 +53,19 @@ In `xce_test`'s worker context, a job dispatched to the fake worker is started b
 - Evidence: not kept (session scratchpad logs)
 
 ## Root cause
-TODO: required for fixed/partial
+Two defects in `xce_test`'s worker context (src/harness/test.c):
+1. Its request handlers (`handle_startjob` -> `run_job`, `handle_runtasks`, `handle_jobcontrol`) run as VTASK_IO tasks of the worker's own connection and called the synchronous notify RPCs directly (`xcecli_notify_jobstart`/`jobfail`/`taskstart`/`taskfail`/`jobchkpt`). Each waits for its reply (notifyAcked, bug-1741), and that reply is dispatched as another task of the same connection, which the vtask manager serialises behind the waiting handler. The call can only time out (-110), and the client then fails over through `xce_ctx_reconnect` (where bug-1917's quiesce hits its 2 s bound, since the caller's own task holds a reference).
+2. After any reconnect the worker session never joined again. The controller treats only the joined connection as a worker, so every later worker request (job rusage `update`, notifications) failed `get_ep_worker` ("is not a worker").
 
 ## Fix
-TODO: required for fixed/partial
+- New `src/harness/worker_notify.{c,h}`: a mutex-protected FIFO of worker notifications (`wn_defer`, `wn_drain`, `wn_pending`) and the re-join predicate `wn_need_rejoin(joined, joined_gen, cur_gen)`. A drain stops at the first failed send and keeps that notification and the rest for the next drain, so a job start is never lost.
+- `test.c`: the notify helpers (`notify_jobstart`, `notify_job_fail`, `notify_task_start`, `notify_task_fail`, and the checkpoint notify in `handle_jobcontrol`) queue the notification and schedule a one-shot flush on the worker's delay-manager thread instead of sending from the handler. `worker_house_keeping` drains the queue on every tick too (retry).
+- `cmd_join_worker` records a successful worker join (connection `reconnect_gen` via `xcecli_get_my_endpoint`, plus a copy of the join attributes). Before draining notifications and before each housekeeping round, `worker_ensure_joined` re-joins with the saved attributes when the connection generation changed.
 
 ## Verification
-- Test: TODO: name + file; RED before the fix, GREEN after
-- Suites: TODO: suite results and build profile
-- Live: TODO: soak/deploy run, or n/a and why
+- Test: `test_worker_notify` (src/harness/testsuite/test_worker_notify.c, harness_unit_test): defer never sends; drain is FIFO with copied payloads; a failed send keeps it and the later ones for the next drain; fini frees what is left (LSan clean); `wn_need_rejoin` truth table. RED before the fix (module absent: link failure), GREEN after.
+- Suites: `xce-build.sh all` asan, 127/127 meson suites OK (harness suite includes test_worker_notify), Go tests ok (2026-10-06).
+- Live: single controller + scheduler + apiserver (`run-resource-verify.sh --keep-running`), one `xce_test` fake worker with a single `join`, 2026-10-06: the job started, the worker never failed over (no -110), all 3 rusage reports were accepted (0 "is not a worker"), and xcecli showed CPU_TIME 100 / MAX_MEM 200 / AVG_MEM 200 / CPU_EFFICIENCY 77.52% (before the fix: every report rejected, all zeros, unless the session joined a second time). Then the controller was killed and restarted: the worker printed "host re-join ok (connection re-established)", the restarted controller ran worker reconciliation for it (a worker join), logged no "is not a worker", and CPU_EFFICIENCY was recomputed to 38.61% (100 s over ~259 s), i.e. a post-restart report was accepted.
 
 ## References
 - Related: none