myxce/.wolf/bugs/bug-1588.md

id	status	status_updated	status_note	component	severity	timestamp	last_seen	occurrences	files	tags	related	legacy
bug-1588	fixed	2026-09-24		controller	unknown	2026-08-19T02:47:00Z	2026-09-19T09:00:00Z	4	src/controller/job.c (dispatch_a_job_ ~841 on_job_start ~1662 on_job_finish ~1766) -- likely more sites not yet grepped exhaustively; PLUS (open not yet fixed) src/worker/src/{xcecd_dbus_manager xcecd main xcecd_worker controller_connector uv_ipc xcecd_cgroups xcecd_worker_utils}.c and src/controller/testsuite/test.c -- see fix field for full enumeration	controller logging observability jobid ux job.c not-yet-investigated worker partial-fix		true

bug-1588: User feedback (UX/observability, not a functional defect), from logs/ctrl-1.log: job IDs printed…

Description

User feedback (UX/observability, not a functional defect), from logs/ctrl-1.log: job IDs printed in controller log lines are large, non-sequential-looking 64-bit numbers (e.g. 4631574334564990976, 4631574334564990977, 4631574334633148416) that are hard to visually scan/compare/diff -- would be better rendered in a more human-readable form. Affected call sites (src/controller/job.c): dispatch_a_job_() (job.c:841, 'dispatch_a_job_ jobid '), on_job_start() (job.c:1662, 'on_job_start jobid '), on_job_finish() (job.c:1766, 'on_job_finish job finish jobid exit code '). Likely other job.c/controller log call sites print the same raw jobid and would benefit from the same treatment (not yet enumerated).

Environment

Not recorded (migrated from buglog.json).

Reproduction

Not recorded (migrated from buglog.json).

Root cause

CONFIRMED, FIXED. The codebase already has a well-established human-readable jobid formatter -- fmt_jobid()/fmt_jobid_ex() (src/lib/utils.c:466-507, declared src/include/utils.h:132-133) -- used consistently across scheduler.c, handler.c, plugins/limit.c, plugins/fcfs.c, repl_info.c (scheduler), process_manager.c, uv_proc.c, xcecd_worker_utils.c (worker), and xce_test/waldump.c (controller testsuite tooling). It renders idgen()'s packed 64-bit jobid (src/lib/idgen.c, bit layout in src/include/config.h: 2 tag bits + 6 cluster-id bits + 4 apiserver-id bits + 32 timestamp-seconds bits + 20 sequence bits, packed via MAKE_JOBID/unpacked via the EXTRACT_ID_* macros in src/include/utils.h) as 'cluster-apiserver-timestamp-sequence' (e.g. '3-1-1755590313-976'), the SAME format REGEX_JOBID/parse_jobid() already parse back from user input and xcecli already displays to end users -- there is no separate 'new' human-readable scheme to design, job.c was simply the one place that never adopted the existing one, printing the raw packed integer via a mismatched %ld (a uint64_t formatted as a signed long -- a latent type/format mismatch, incidentally fixed as a side effect since the fix replaces the whole format specifier with %s).

Fix

Fixed. src/controller/job.c: all 8 TRACE() call sites that printed a raw jobid now call fmt_jobid(jobidStr, sizeof(jobidStr), ) into a local char jobidStr[64] (matching scheduler.c's established buffer-size convention) immediately before the TRACE() call, and print %s instead of %ld: dispatch_a_job_() (~841), on_job_start() (~1662), on_job_finish() (~1766), on_job_fail() (~2026), on_job_stop() (~2055), on_job_resume() (~2090), on_job_postdone() (~2123), set_preason() (~2297). fmt_jobid is already transitively visible in job.c via request.h's #include "utils.h" -- no new include needed. Purely a log-formatting change, zero logic/behavior change -- verified via /xce-build controller asan --incremental: job.c compiles with zero errors/warnings (compiled twice, for xce_controller and controller_unit_test), full controller suite green (1104/1104 Unity assertions, 5/5 meson test cases), git diff confirms job.c is the only file touched in src/controller/. Not live-validated against a real deployment's log output (no functional/behavioral risk to validate beyond the build+test suite, since this only changes what a string looks like) -- if desired, a live run would show e.g. 'dispatch_a_job_ jobid 3-1-1755590313-976' instead of 'dispatch_a_job_ jobid 4631574334564990976'.

---

Remaining sites enumerated 2026-08-21 via ag -i job src | grep '%lu' (job.c's 8 sites from this bug are already fixed; these are NOT yet fixed):

WORKER (src/worker/src/) -- still print the raw uint64 jobid via %lu instead of calling fmt_jobid():

• xcecd_dbus_manager.c:423
• xcecd.c:60
• main.c:167,227,245
• xcecd_worker.c: ~25 sites between lines 83-1314 (rusage, VNC, exdata, port assignment, signal/pause/resume, CRIU checkpoint status messages)
• controller_connector.c: ~20 sites between lines 508-1857 (job dispatch, remote task, job-control action, rusage-to-controller messages)
• uv_ipc.c: ~20 sites between lines 77-1541 (client mapping, signal/checkpoint ack, rusage, job completion/failure)
• xcecd_cgroups.c:1703 (rusage summary log line; xcecd_cgroups.c:1405's xcecd-worker-%lu.service unit name and xcecd_worker.c:1209's %s/%lu checkpoint dir / xcecd_exdata.c:408's numeric-string exdata substitution are identifier/path construction, not human-facing log text -- left as-is pending a decision on whether those should change too)
• xcecd_worker_utils.c: 8 sites between lines 240-504 -- correction to this bug's original note: original text claimed xcecd_worker_utils.c "already use fmt_jobid() consistently", but grep -c fmt_jobid( src/worker/src/xcecd_worker_utils.c now returns 0; that claim does not hold for the current file state (process_manager.c and uv_proc.c's fmt_jobid() usage, also cited in the original note, is still confirmed correct)

CONTROLLER TESTSUITE (src/controller/testsuite/test.c) -- interactive test-tool printf() output, lower priority than production daemon logs:

• lines 2599, 2649, 3966, 4043, 4048, 4053, 4155, 4180, 4294, 4353

Explicitly OUT of scope -- these matched the %lu grep but are not job IDs, so must NOT be converted: src/controller/job.c:1332 and src/controller/persist.c:6376,6530 all format lsn (WAL log sequence number), not a jobid; src/worker/src/main.c:160 formats a client ID, not a jobid.

Suggested fix pattern (matches this bug's job.c fix): at each site, add char jobidStr[64]; fmt_jobid(jobidStr, sizeof(jobidStr), <jobid_var>); immediately before the log call, and replace the %lu for that argument with %s / jobidStr. utils.h (which declares fmt_jobid) needs to be confirmed reachable in each of these worker files (uv_ipc.c, controller_connector.c, xcecd_worker.c, xcecd_dbus_manager.c, xcecd.c, main.c, xcecd_worker_utils.c) before starting -- not yet verified.

Not yet fixed. Logged as a follow-up scope expansion of this same bug per user request ("log an issue to fix all job ids which are shown as uint64 to human readable string via fmt_jobid()"), 2026-08-21.

--- FOLLOW-UP FIXED 2026-08-21 ---

All remaining sites enumerated above are now fixed too (user request: "make sure that all job ids which have been shown will use human readable format ... please also include src/controller/testsuite/test.c, as we use it test controller heavily"). Same fmt_jobid()+%s pattern as job.c, applied via 5 parallel edit passes (one per file/group) each independently instructed with the exact site list and the exclusion list below, then independently reviewed line-by-line against the originals by the orchestrating session (not just trusted on the subagents' self-reports) before building.

Files fixed (99 call sites total across 9 files):

• src/worker/src/xcecd_worker.c -- 30 sites (worker_signal_callback needed two buffers, jobidStr + curJobidStr, for its two distinct job-id values on one line)
• src/worker/src/controller_connector.c -- 21 sites (handle_fetchfile: only the jobid= half of a two-%lu line converted, offset= left alone)
• src/worker/src/uv_ipc.c -- ~20 sites (add_job_client_mapping: only the job_id half of a two-%lu line converted, client->id left alone)
• src/worker/src/xcecd_worker_utils.c -- 9 sites
• src/worker/src/controller_connector.c, xcecd_dbus_manager.c, xcecd.c, main.c, xcecd_cgroups.c -- remaining small sites (main.c's 4 sites all genuine job ids despite one message's misleading "client ID" wording -- verified against the function parameter type/usage, not just the string)
• src/controller/testsuite/test.c -- 10 sites, matching this file's own pre-existing fmt_jobid()-with-'- 1'-sizeof convention

Deliberately left unconverted (confirmed non-jobid or non-log-text, do not touch):

• src/controller/job.c:1332 and src/controller/persist.c:6376,6530 -- format lsn (WAL log sequence number), not a jobid
• src/worker/src/controller_connector.c:1376 -- the offset=%lu half of a fetchfile log line
• src/worker/src/uv_ipc.c -- the client->id half of add_job_client_mapping's log line
• src/worker/src/xcecd_worker.c:1231 (ckpt_dir path), xcecd_cgroups.c:1405 (xcecd-worker-%lu.service unit name), xcecd_exdata.c:408 (XCE_JOBID env var set for the job's own process) -- all raw-numeric identifier/path/protocol construction, not human-facing log text; changing these would be a behavioral change (on-disk paths, systemd unit names, a documented env var format), not a log-formatting change

Verification:

• /xce-build worker asan --incremental: SUCCESS, 430/430 ninja targets, zero warnings/errors related to fmt_jobid/jobidStr in any of the 8 edited worker files, worker meson test suite 4/4 OK (resource_sync, xcecd_startup_order, dbus_workers_update, controller_connector tests).
• /xce-build controller asan --incremental: SUCCESS, test.c compiled with zero warnings/errors, controller meson test suite 4/5 OK -- the 1 failure (test_config_repl.c, 2 Unity assertions) is the pre-existing, previously-logged bug-1561 overlayfs-sandbox fdatasync flake (7th occurrence, git diff --stat confirms config_repl.c/test_config_repl.c untouched by this change) -- not caused by this fix.
• Every diff was read and checked line-by-line against the pre-edit originals by the orchestrating session (argument-to-specifier order, buffer scoping/reuse correctness, exclusion list respected) before either build ran -- not just accepted on the editing subagents' self-reports.

Not fixed / explicitly out of scope: no attempt was made to reconcile the on-disk checkpoint-dir/systemd-unit-name/XCE_JOBID raw-numeric conventions with fmt_jobid() -- flagged above as a separate judgment call if ever wanted, not part of this fix.

Verification

Not recorded (migrated from buglog.json).

References

Not recorded (migrated from buglog.json).