diff --git a/.wolf/buglog.json b/.wolf/buglog.json index 0ba8e4b4..04237639 100644 --- a/.wolf/buglog.json +++ b/.wolf/buglog.json @@ -27844,6 +27844,49 @@ "related_bugs": [], "occurrences": 1, "last_seen": "2026-09-24T13:02:14Z" + }, + { + "id": "bug-1858", + "status": "fixed", + "status_updated": "2026-09-24", + "status_note": "393a07c2", + "timestamp": "2026-09-24T13:12:53Z", + "error_message": "AddressSanitizer: attempting free on address which was not malloc()-ed in core_upsert_queue (core_events.c:383) <- core_apply_config_queues <- xce_core_push_events <- sched_sim_driver_run; address 56 bytes inside a 57-byte makeString instance. Seen as HA soak (sequential) FAIL: scheduler replay-parity SIMVERIFY errors=1 (simverify-3, rc=1)", + "file": "src/scheduler/src/core_events.c,src/scheduler/src/schedobj.c", + "root_cause": "q->resreq is an interned, refcounted xstring from makeString() (pointer past the object header), but core_upsert_queue releases the old value with FREEUP()/free() instead of dereference(). Only fires when a CONFIG_PUSH updates an existing queue whose resreq is non-NULL (e.g. a queue restored from snapshot via snapshot_load.c:313). Same bad free(q->resreq) exists in the normal-mode path, schedobj.c:603 (sched_update_existing_queue). Convention elsewhere: qobj.c:69 dereference(q->queue.resreq).", + "fix": "UPDATE (2026-09-24): Fix (uncommitted): core_events.c core_upsert_queue and schedobj.c sched_update_existing_queue now dereference() the old interned q->resreq (NULL-guarded) instead of FREEUP()/free(). Regression tests added (red first, both hit the exact bad-free): test_core_events.c test_queue_update_replaces_existing_interned_resreq, test_job_update.c test_update_existing_queue_replaces_interned_resreq (the existing queue-update test had deliberately skipped the resreq branch). Verified 2026-09-24: scheduler suite 87/87 (ASAN); the crashing HA-soak recording scheduler-replay-20260924-130653.replay now replays with rc=0, 0 ASAN errors, 0 mismatches.", + "tags": [ + "scheduler", + "asan", + "sim", + "bad-free", + "soak" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-09-24T13:12:53Z" + }, + { + "id": "bug-1859", + "status": "investigating", + "status_updated": "2026-09-24", + "status_note": "chaotic soak: worker-resync job stuck PEND (ctrl) vs RUN (sched) after overlapping failover; 1 of 2 chaotic runs", + "timestamp": "2026-09-24T13:45:31Z", + "error_message": "xce-test-ha --chaotic FAIL: chaotic: 1/4 worker-resync job(s) never reconciled after the loop settled: 0-2-1790256202-546579(last=PEND). scheduler.log: 'JOBS_PUSH: refusing resync state 0x1 for jobid ... -- existing job is RUN with a real allocation on record (existing state 0x2 kept)'", + "file": "src/scheduler/src/core_events.c,src/controller/job.c", + "root_cause": "Not yet confirmed. Timeline (2026-09-24 chaotic soak, 6 rounds): job dispatched on primary ctrl-1 (term 6) at 13:23:23.74, commit_advance 80->81 at 13:23:23.79; worker started it and finished it at 13:23:43 (retained outcome exit_status=1); ctrl-1 killed in round 5, overlapping kills follow; after settle the cluster reports the job PEND while the scheduler keeps RUN+alloc and refuses the PEND resync (core_events.c:984), so neither side converges. Also: round 6 'scheduler restart did not confirm join within 20s (bug-1760 workaround)'. Candidates: dispatch/RUN not in the surviving committed prefix while the scheduler already applied it, or worker's retained finish never re-delivered to the new primary (cf. bug-1741 fire-and-forget finish notify). Same chaotic soak passed earlier the same day (21 jobs, 6 resync jobs reconciled), so intermittent.", + "fix": "", + "tags": [ + "ha", + "chaotic-soak", + "resync", + "scheduler", + "controller", + "intermittent" + ], + "related_bugs": [], + "occurrences": 1, + "last_seen": "2026-09-24T13:45:31Z" } ] }