Rongsong Shen
d26ff8870d
The buglog was one 27K-line JSON file whose fields covered only symptom,
root cause and fix, as free text: 2% of entries named the code version a
bug was found on, 12% the build profile, 36% a way to reproduce, and 58% of
the file was OpenWolf hook auto-detections ("Missing guard clause",
"Function not marked async" on .md files).
- .claude/skills/buglog/bug-record.md: the record standard. Eight
sections (Summary as the title, Description, Environment, Reproduction;
Root cause, Fix, Verification, References once fixed), each with the
content it must contain -- e.g. Environment needs the OS, the libc the
failing binary loads (from its ELF interpreter: glibc 2.42 in the nix
shell vs 2.40 on the host), toolchain, build profile and commit.
- .wolf/bugs/<id>.md: 500 records migrated from buglog.json; every field
verified to round-trip. They carry legacy: true and an explicit "not
recorded" where the old log had no such field; component derived from
file paths, severity from keywords (else unknown), fix_commits from
hashes in status_note/fix that git resolves (91 of 463 fixed).
- .wolf/buglog-archive.json: the 710 hook auto-detections, verbatim.
- .wolf/buglog.json is now an index generated from the records, keeping
the fields OpenWolf's hooks and `openwolf bug search` read.
- scripts/buglog.py rewritten: new (template + Environment collected from
the machine and --binary), check, status (refuses a status the record
doesn't support; fixed needs Root cause, Fix, Verification and
--commit hashes git resolves), note, recur, set, show, list, triage,
env, index. 29 unittest cases in scripts/tests/test_buglog.py.
- scripts/git-hooks/pre-commit runs `check --staged`: incomplete staged
records, a legacy record hand-changed to fixed without the fix-time
sections, a new record claiming legacy: true, or a stale index refuse
the commit.
- scripts/gitea-sync.py reads the records: issue body = the record's
sections plus linked fix commits (origin remote, GITEA_COMMIT_REPO to
override), and a body hash so an issue's text is refreshed when the
record changes.
- bug-1866 rewritten to the full standard as the worked example.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
86 lines
10 KiB
Markdown
86 lines
10 KiB
Markdown
---
|
|
id: "bug-1588"
|
|
status: "fixed"
|
|
status_updated: "2026-09-24"
|
|
status_note: ""
|
|
component: "controller"
|
|
severity: "unknown"
|
|
timestamp: "2026-08-19T02:47:00Z"
|
|
last_seen: "2026-09-19T09:00:00Z"
|
|
occurrences: 4
|
|
files: ["src/controller/job.c (dispatch_a_job_ ~841", "on_job_start ~1662", "on_job_finish ~1766) -- likely more sites", "not yet grepped exhaustively; PLUS (open", "not yet fixed) src/worker/src/{xcecd_dbus_manager", "xcecd", "main", "xcecd_worker", "controller_connector", "uv_ipc", "xcecd_cgroups", "xcecd_worker_utils}.c and src/controller/testsuite/test.c -- see fix field for full enumeration"]
|
|
tags: ["controller", "logging", "observability", "jobid", "ux", "job.c", "not-yet-investigated", "worker", "partial-fix"]
|
|
related: []
|
|
legacy: true
|
|
---
|
|
|
|
# bug-1588: User feedback (UX/observability, not a functional defect), from logs/ctrl-1.log: job IDs printed…
|
|
|
|
## Description
|
|
User feedback (UX/observability, not a functional defect), from logs/ctrl-1.log: job IDs printed in controller log lines are large, non-sequential-looking 64-bit numbers (e.g. `4631574334564990976`, `4631574334564990977`, `4631574334633148416`) that are hard to visually scan/compare/diff -- would be better rendered in a more human-readable form. Affected call sites (src/controller/job.c): `dispatch_a_job_()` (job.c:841, 'dispatch_a_job_ jobid <id>'), `on_job_start()` (job.c:1662, 'on_job_start jobid <id>'), `on_job_finish()` (job.c:1766, 'on_job_finish job finish jobid <id> exit code <n>'). Likely other job.c/controller log call sites print the same raw jobid and would benefit from the same treatment (not yet enumerated).
|
|
|
|
## Environment
|
|
_Not recorded (migrated from buglog.json)._
|
|
|
|
## Reproduction
|
|
_Not recorded (migrated from buglog.json)._
|
|
|
|
## Root cause
|
|
CONFIRMED, FIXED. The codebase already has a well-established human-readable jobid formatter -- fmt_jobid()/fmt_jobid_ex() (src/lib/utils.c:466-507, declared src/include/utils.h:132-133) -- used consistently across scheduler.c, handler.c, plugins/limit.c, plugins/fcfs.c, repl_info.c (scheduler), process_manager.c, uv_proc.c, xcecd_worker_utils.c (worker), and xce_test/waldump.c (controller testsuite tooling). It renders idgen()'s packed 64-bit jobid (src/lib/idgen.c, bit layout in src/include/config.h: 2 tag bits + 6 cluster-id bits + 4 apiserver-id bits + 32 timestamp-seconds bits + 20 sequence bits, packed via MAKE_JOBID/unpacked via the EXTRACT_ID_* macros in src/include/utils.h) as 'cluster-apiserver-timestamp-sequence' (e.g. '3-1-1755590313-976'), the SAME format REGEX_JOBID/parse_jobid() already parse back from user input and xcecli already displays to end users -- there is no separate 'new' human-readable scheme to design, job.c was simply the one place that never adopted the existing one, printing the raw packed integer via a mismatched `%ld` (a uint64_t formatted as a signed long -- a latent type/format mismatch, incidentally fixed as a side effect since the fix replaces the whole format specifier with %s).
|
|
|
|
## Fix
|
|
Fixed. src/controller/job.c: all 8 TRACE() call sites that printed a raw jobid now call fmt_jobid(jobidStr, sizeof(jobidStr), <jobid>) into a local `char jobidStr[64]` (matching scheduler.c's established buffer-size convention) immediately before the TRACE() call, and print `%s` instead of `%ld`: dispatch_a_job_() (~841), on_job_start() (~1662), on_job_finish() (~1766), on_job_fail() (~2026), on_job_stop() (~2055), on_job_resume() (~2090), on_job_postdone() (~2123), set_preason() (~2297). fmt_jobid is already transitively visible in job.c via request.h's #include "utils.h" -- no new include needed. Purely a log-formatting change, zero logic/behavior change -- verified via /xce-build controller asan --incremental: job.c compiles with zero errors/warnings (compiled twice, for xce_controller and controller_unit_test), full controller suite green (1104/1104 Unity assertions, 5/5 meson test cases), git diff confirms job.c is the only file touched in src/controller/. Not live-validated against a real deployment's log output (no functional/behavioral risk to validate beyond the build+test suite, since this only changes what a string looks like) -- if desired, a live run would show e.g. 'dispatch_a_job_ jobid 3-1-1755590313-976' instead of 'dispatch_a_job_ jobid 4631574334564990976'.
|
|
|
|
---
|
|
|
|
Remaining sites enumerated 2026-08-21 via `ag -i job src | grep '%lu'` (job.c's 8 sites from this bug are already fixed; these are NOT yet fixed):
|
|
|
|
WORKER (src/worker/src/) -- still print the raw uint64 jobid via %lu instead of calling fmt_jobid():
|
|
- xcecd_dbus_manager.c:423
|
|
- xcecd.c:60
|
|
- main.c:167,227,245
|
|
- xcecd_worker.c: ~25 sites between lines 83-1314 (rusage, VNC, exdata, port assignment, signal/pause/resume, CRIU checkpoint status messages)
|
|
- controller_connector.c: ~20 sites between lines 508-1857 (job dispatch, remote task, job-control action, rusage-to-controller messages)
|
|
- uv_ipc.c: ~20 sites between lines 77-1541 (client mapping, signal/checkpoint ack, rusage, job completion/failure)
|
|
- xcecd_cgroups.c:1703 (rusage summary log line; xcecd_cgroups.c:1405's `xcecd-worker-%lu.service` unit name and xcecd_worker.c:1209's `%s/%lu` checkpoint dir / xcecd_exdata.c:408's numeric-string exdata substitution are identifier/path construction, not human-facing log text -- left as-is pending a decision on whether those should change too)
|
|
- xcecd_worker_utils.c: 8 sites between lines 240-504 -- correction to this bug's original note: original text claimed xcecd_worker_utils.c "already use fmt_jobid() consistently", but `grep -c fmt_jobid( src/worker/src/xcecd_worker_utils.c` now returns 0; that claim does not hold for the current file state (process_manager.c and uv_proc.c's fmt_jobid() usage, also cited in the original note, is still confirmed correct)
|
|
|
|
CONTROLLER TESTSUITE (src/controller/testsuite/test.c) -- interactive test-tool printf() output, lower priority than production daemon logs:
|
|
- lines 2599, 2649, 3966, 4043, 4048, 4053, 4155, 4180, 4294, 4353
|
|
|
|
Explicitly OUT of scope -- these matched the `%lu` grep but are not job IDs, so must NOT be converted: src/controller/job.c:1332 and src/controller/persist.c:6376,6530 all format `lsn` (WAL log sequence number), not a jobid; src/worker/src/main.c:160 formats a `client ID`, not a jobid.
|
|
|
|
Suggested fix pattern (matches this bug's job.c fix): at each site, add `char jobidStr[64]; fmt_jobid(jobidStr, sizeof(jobidStr), <jobid_var>);` immediately before the log call, and replace the `%lu` for that argument with `%s` / jobidStr. utils.h (which declares fmt_jobid) needs to be confirmed reachable in each of these worker files (uv_ipc.c, controller_connector.c, xcecd_worker.c, xcecd_dbus_manager.c, xcecd.c, main.c, xcecd_worker_utils.c) before starting -- not yet verified.
|
|
|
|
Not yet fixed. Logged as a follow-up scope expansion of this same bug per user request ("log an issue to fix all job ids which are shown as uint64 to human readable string via fmt_jobid()"), 2026-08-21.
|
|
|
|
--- FOLLOW-UP FIXED 2026-08-21 ---
|
|
|
|
All remaining sites enumerated above are now fixed too (user request: "make sure that all job ids which have been shown will use human readable format ... please also include src/controller/testsuite/test.c, as we use it test controller heavily"). Same fmt_jobid()+%s pattern as job.c, applied via 5 parallel edit passes (one per file/group) each independently instructed with the exact site list and the exclusion list below, then independently reviewed line-by-line against the originals by the orchestrating session (not just trusted on the subagents' self-reports) before building.
|
|
|
|
Files fixed (99 call sites total across 9 files):
|
|
- src/worker/src/xcecd_worker.c -- 30 sites (worker_signal_callback needed two buffers, jobidStr + curJobidStr, for its two distinct job-id values on one line)
|
|
- src/worker/src/controller_connector.c -- 21 sites (handle_fetchfile: only the jobid= half of a two-%lu line converted, offset= left alone)
|
|
- src/worker/src/uv_ipc.c -- ~20 sites (add_job_client_mapping: only the job_id half of a two-%lu line converted, client->id left alone)
|
|
- src/worker/src/xcecd_worker_utils.c -- 9 sites
|
|
- src/worker/src/controller_connector.c, xcecd_dbus_manager.c, xcecd.c, main.c, xcecd_cgroups.c -- remaining small sites (main.c's 4 sites all genuine job ids despite one message's misleading "client ID" wording -- verified against the function parameter type/usage, not just the string)
|
|
- src/controller/testsuite/test.c -- 10 sites, matching this file's own pre-existing fmt_jobid()-with-'- 1'-sizeof convention
|
|
|
|
Deliberately left unconverted (confirmed non-jobid or non-log-text, do not touch):
|
|
- src/controller/job.c:1332 and src/controller/persist.c:6376,6530 -- format `lsn` (WAL log sequence number), not a jobid
|
|
- src/worker/src/controller_connector.c:1376 -- the `offset=%lu` half of a fetchfile log line
|
|
- src/worker/src/uv_ipc.c -- the `client->id` half of `add_job_client_mapping`'s log line
|
|
- src/worker/src/xcecd_worker.c:1231 (ckpt_dir path), xcecd_cgroups.c:1405 (`xcecd-worker-%lu.service` unit name), xcecd_exdata.c:408 (`XCE_JOBID` env var set for the job's own process) -- all raw-numeric identifier/path/protocol construction, not human-facing log text; changing these would be a behavioral change (on-disk paths, systemd unit names, a documented env var format), not a log-formatting change
|
|
|
|
Verification:
|
|
- /xce-build worker asan --incremental: SUCCESS, 430/430 ninja targets, zero warnings/errors related to fmt_jobid/jobidStr in any of the 8 edited worker files, worker meson test suite 4/4 OK (resource_sync, xcecd_startup_order, dbus_workers_update, controller_connector tests).
|
|
- /xce-build controller asan --incremental: SUCCESS, test.c compiled with zero warnings/errors, controller meson test suite 4/5 OK -- the 1 failure (test_config_repl.c, 2 Unity assertions) is the pre-existing, previously-logged bug-1561 overlayfs-sandbox fdatasync flake (7th occurrence, `git diff --stat` confirms config_repl.c/test_config_repl.c untouched by this change) -- not caused by this fix.
|
|
- Every diff was read and checked line-by-line against the pre-edit originals by the orchestrating session (argument-to-specifier order, buffer scoping/reuse correctness, exclusion list respected) before either build ran -- not just accepted on the editing subagents' self-reports.
|
|
|
|
Not fixed / explicitly out of scope: no attempt was made to reconcile the on-disk checkpoint-dir/systemd-unit-name/XCE_JOBID raw-numeric conventions with fmt_jobid() -- flagged above as a separate judgment call if ever wanted, not part of this fix.
|
|
|
|
## Verification
|
|
_Not recorded (migrated from buglog.json)._
|
|
|
|
## References
|
|
_Not recorded (migrated from buglog.json)._
|